EXT: Direct Mail

Table of Contents

Copyright	2

Credits	2

Sponsors	2

Introduction	3

What does it do?	3

Requirements	3

Users Manual	4

Summary of the Direct Mail process	4

Accessing the Direct Mail module	4

The Direct Mail Wizard	5

Personalizing direct mail content	8

Types of recipient lists	9

Importing a csv list of email addresses	9

Configuration	12

Installing the extension	12

Creating a Direct Mail folder	12

Configuring the Direct Mail module in Page TSConfig	13

Configuring the Direct Mail Wizard in User TSConfig	16

Configuring the Calendar of the Direct Mail Wizard	16

Configuring the TS template for newsletter pages	16

Configuring plain text rendering	17

Configuring plain text rendering of News records	18

Configuring the use of categories	19

Enabling click statistics	19

Configuring the mailer engine cron task	20

Using Sendmail deferred delivery mode	20

Configuring the analysis of returned mails	20

Configuring SMTP	21

Upgrading to version 2.0+	22

Upgrading to version 2.5.0	22

Troubleshooting	23

Direct mails are not sent	23

Mass mail is only delivered after sending another mail from same MAC OS X server	23

Known Problems	24

To-Do List	25

Change Log	26

Copyright

Extension Key: direct_mail

Copyright 2006, , <ivan.kartolo@dkd.de>

This document is published under the Open Content License

available from http://www.opencontent.org/opl.shtml

The content of this document is related to TYPO3

a GNU/GPL CMS/Framework available from www.typo3.com

Credits

Thanks to Kasper Skårhøj, original author of this extension, and to Stanislas Rolland, Jan-Erik Revsbech, Thorsten Kahler, Martin Kutschker, Robert Böttner, Andreas Schwarzkopf, Bertrand Mure, Urs Weiss, Rik Hoevers, Robert Lemke, Florian Bittner, Ralf Hettinger and David Bocher for their contributions to its evolution.

Sponsors

You may sponsor further development of this extension by contacting the author of this document.

Introduction

What does it do?

The Direct Mail extension is an advanced newsletter mailer system with sophisticated options for personalization of emails including response statistics.

This extension handles the backend generation and mailing of newsletters, but does not provide any subscription tool. It is intended to be use in conjunction with frontend subscription plugins such as Direct Mail Subscription (direct_mail_subscription), Front End User Registration (sr_feuser_register) and Email Address Subscription (sr_email_subscribe).

Requirements

Extension Direct Mail (extension key: direct_mail) requires TYPO3 4.0.0+ and PHP 4.1.0+.

Version 1.1.0 is the last version of this extension that will work with TYPO3 3.8.

Extension Address list (tt_address) is a prerequisite.

Users Manual

Summary of the Direct Mail process

The Direct Mail process let you create newsletter pages which can then be emailed to people on a recipient list. The following are the main steps of the Direct mail process:

Creating a newsletter: a newsletter is basically a regular Typo3 page which resides in the Direct Mail folder; you can view the page in a browser and it may also be rendered in plain text.

Categorizing the content elements: you may assign categories to the content elements of the newsletter; the newsletter emailed to each subscriber will be tailored to the categories the subscriber has subscribed to.

Creating a direct mail: a direct mail is a record that contains a compiled version of either a newsletter page or alternatively the content of an external url. In addition, the direct mail contains information like the mail subject, any attachments, priority settings, reply addresses , etc. For each direct mail, a log is kept of who has received the direct mail and if they responded to it.

Building a recipient list: there is a number of ways by which recipient lists may be built. Recipient lists may be built beforehand and be reused as needed.

Sending a test: it is good practice to send a test mail to make sure that the email messages that will be sent to the recipient list look as expected when received in a client email application.

Invoking the mailer engine: schedule the actual sending of the direct mail to a specific recipient list.

Viewing the status of mass mailings: monitor the sending of the direct mails, view response statistics, and take action on returned mails.

At any point, you may, if needed, use the module context sensitive help. Backend forms for editing direct mails and recipient lists also provide context sensitive help on each field.

Accessing the Direct Mail module

Upon the installation of the Direct Mail extension there will be a new section, called “Direct Mail” in the backend menu. In this section you will find 5 modules:

Direct Mail: with this module you can send direct mail based on a new newsletter (internal page), an external page, a quick mail or an existing direct mail. The sending process is a four or five steps wizard.

Recipients Lists: this function lets you create a new recipient list, import Addresses records from csv data to create a new recipient list, link to a recipient list editing form, or select an existing recipient list; selecting an existing recipient list leads you to a screen that lets you view the number of recipients of each type in the list, with the options to list the recipients or download a csv file. The import process is a wizard. You can upload a CSV file and freely mapping the fields

Statistics: it shows a list of sent direct mails. If you choose a direct mail, the detailed statistics will be shown. You can also list, disable or download the list of recipients, whose mails are returned.

Mailer Engine: shows the queue of the mailer engine and the status of the cron job (if it's set). You can also manually invoke the engine by clicking the link. This link can be hidden per UserTS or PageTS.

Configuration: in this module you can configure the direct mail. The configuration is saved in the Direct Mail SysFolder. It means that every SysFolder has a different configuration. In this module there a second function menu, the Categories Conversion. This function is only used if you're upgrading to Direct Mail version 2.0.0. With the introduction of version 2.0.0 Direct Mail saves the Categories in a table.

The Direct Mail Wizard

The Direct Mail Wizard is located in the first module (Direct Mail). After choosing a Direct Mail SysFolder, the first step is shown. Four possible sources of a Direct Mail is shown: internal page (TYPO3 page), external page, quickmail and from existing Direct Mail.

Internal Page

All page in the selected SysFolder will be listed. There will be 3 icons, edit icon and two preview icon. With the edit icon you can edit the page contents. The first preview icon shows the HTML-version and the second one shows the Plaintext-version of the direct mail. Beside that you can create a new page by clicking the last link.

By clicking the page title, you choose a page as a direct mail and the second step will be shown.

External Page

To create a new direct mail based on an external page, you should insert the HTML and Plaintext URL of the external page and the subject of the direct mail. To continue to the second step click the “create mail” button

Quickmail

Default values for the sender name and email are the values, which you entered in the configuration module. You can only send plaintext version. In the second step you can put attachments like a normal direct mail.

Direct Mail

All existing and not yet sent direct mail are listed in this option. You can choose an existing direct mail to continue the sending process by clicking the subject of the direct mail.

In the second step, the detail information of the direct mail is shown and you can manually change this, e.g. insert attachments, by clicking the edit button. In this step the content of of the direct mail is automatically fetched and an information on the fetching is also shown (successful, warning or error). If there is an error, you should leave the wizard , correct this error (e.g. by clearing FE-cache) and continue the sending process by choosing from the list of existing Direct Mail (fourth Option in the first step).

The third step is the categorizing of the content. This step will be shown if the direct mail's source is a TYPO3 step. If the direct mail's source is other than TYPO3 page, then it will be skipped to step four. After choosing the categories, click the “update category settings” button to save the assignment of the categories.

To continue the sending process, click the next button. The fourth step will be shown.

The fourth step is sending test mails. If UIDs of tt_address or direct mail groups in the configuration module are given, they will be shown and click the name will send email to them and this step will be shown again. If there is no UID given in the configuration module, there will be only a text field, where you can give an email address.

After finish testing the direct mail, click on the next button will show the fifth and final step.

The last step is mass sending the direct mail. After choosing a recipient lists from a drop down lists, you can choose time, when the mailer engine start sending it. The button next to the input field will show a pop up calendar. You can choose date and time. By clicking the “send to all subscribers in the recipient list” button, the direct mail is released to sending

Personalizing direct mail content

You may personalize the direct mails that will be sent to each recipient by inserting the following markers in content elements of the newsletter page. The markers will be substituted with the corresponding value found in each recipient record, whenever available. This is the list of predefined markers:

###USER_uid### (the unique id of the recipient)

###USER_name### (full name)

###USER_firstname### (first name calculated)

###USER_NAME### and ###USER_FIRSTNAME### will insert uppercase versions of the equivalents

###USER_title###

###USER_email###

###USER_phone###

###USER_www###

###USER_address###

###USER_company###

###USER_city###

###USER_zip###

###USER_country###

###USER_fax###

###SYS_TABLE_NAME###

###SYS_MAIL_ID###

###SYS_AUTHCODE###

Additional fields from the recipient table (e.g. defined by an extension) can be configured in the Extension Manager. You have to switch to "Information" view of EXT:direct_mail and set a comma-separated list of DB fields in "Additional DB fields of the recipie..." (addRecipFields) to make more markers available. These can be used according to the pattern above like ###USER_<some field>###.

Personalization only works for recipient lists of type "From pages" (and "From other recipient lists", if these contain "From pages" lists).

Types of recipient lists

There are five types of Recipient lists:

From pages: a list dynamically built from records found on pages of the website; you select the pages with the Starting Point field; you may include records from subtrees using the Include page subtree checkbox; you must specify the types of records to select: Address, Website User, Website User Group and/or From custom-defined Table; if Website User Group is checked and a Website User Group is found, all Website Users members of the group will be included in the list; you may also restrict the selection of records to those with specific categories assignments;

Plain list: a list of email addresses separated by space, comma or line break; if you change the List format, you may rather enter, one per line, a name and address, separated by a comma; sending Direct mail to such list will send only plain text email messages; moreover, only non-categorized content will be sent;

Static list: a list of individually selected records; the type of record may be Address, Website User, and/or Website User Group; if a Website User Group is selected, all Website Users members of the group will be included in the list;

Special query: a list dynamically built using a SQL query; after creating the list, you may specify the query against one of the tables: Address, Website User or Custom-defined Table; to do so, you must select the Recipient list function of the Direct Mail module and click on the title of the list;

From other recipient lists: a list dynamically built from other recipient lists individually selected from the Recipient list table.

For more information on the individual fields found on the creation/editing form of recipient lists, please use the contextual help attached to each such field.

Importing a csv list of email addresses

The Recipient Lists module gives you the option to import a csv (comma-separates list) of address records and create a recipient list containing the imported records.

To make it easier to import csv records, there is a wizard, which guide you through the process.

In the first step you can choose to upload a csv file or paste the records into a text field. You can use comma (;), semicolon (;), or colon (:) as field delimiter. This can be configured in the second step.

By clicking the next button, the csv file or csv records are uploaded and the second step is shown. In this step you can specify the detail information of the csv data, such as field delimiter, field encapsulation, and field name in the first line. You can also specify the SysFolder, where the records should be imported to, the uniqueness of the records, rename or update the records if a similar record is found, or to empty the SysFolder before importing.

ATTENTION! If you set the field “remove all Addresses in the storage folder before importing”, all records in this SysFolder WILL be physically deleted.

After specifying the configuration you can start mapping the fields. There are 3 columns in the mapping step. The description column shows the first row of the csv records (if you set in the configuration that the first row is the field names) or shows only field_xx (where xx is continuous number).

The mapping column shows only the list of field, which are part of tt_address table. You must at least map the field “Name” and “Email”.

The value column shows the first up to three rows from the csv records. They should help you to map the field.

After mapping the fields you are ready to start the import process. To start the import click the import button. After the importing, a list of new imported, invalid email, updated and doublet records will be shown.

Configuration

Installing the extension

Import and install the extension using the Extension Manager.

You may be required to install extension Address list (tt_address) which is a prerequisite of extension Direct Mail.

Some updates to the database structure may be needed. Press the “Make updates” button to make these updates.

When the extension is installed, a “Direct Mail” Section and five new modules will appear above the Help section of the backend menu, provided that the user is granted access using the User Admin tool.

The Extension Manager installation dialog allows to set the following extension configuration variables:

Number of messages per cycle: number of messages sent per cycle of the mailing engine cron task; default value is 50;

Language of the cron task: the TYPO3 language code of the language to be used by the mailing engine cron task when progress messages are sent to the administrator; default value is “en”;

Additional DB fields of the recipient: additional fields that may be substituted in the direct mail messages;

Administrator Email: Email will be sent to this address, if there is an error in the cron task. Especially to notify administrator of the site if the table can't be written (Direct Mail creates a lot of log records while sending the email).

Interval of the cronjob: Time interval of the cron task you set in minute. This is used to control if the cron task is still running or if there is an error while sending the email.

Enable notification email: Allow Direct Mail to send notification emails about start and end of mailing job.

Enable Sendmail defer mode: use Sendmail deferred delivery mode; applies only to the Sendmail program on Unix/Linux; default value is 0. Note that the option cannot be set if PHP is running with safe_mode enabled;

Enable plain text rendering of News: if set, a script will be enabled to render News (tt_news records) in plain text for inclusion as plain text content of email messages.

Enable Sending through SMTP: Send all emails not through sendmail but through an external SMTP account (requires PEAR::Mail to be installed on the system)

SMTP Host: Name or IP of the SMTP host, only used when sending through SMTP is enabled

SMTP Port: Port for the SMTP protocol of the SMTP host, only used when sending through SMTP is enabled

SMTP authentication: Activate if your SMTP account needs Authentication, only used when sending through SMTP is enabled

SMTP Username: Use it if your SMTP account needs a authentication, only used when sending through SMTP is enabled

SMTP Password: Use it if your SMTP account needs a authentication, only used when sending through SMTP is enabled

SMTP persistent connection: Activate if you want to use a persistent connection while sending direct mails, only used when sending through SMTP is enabled

If you are upgrading to version 2.0+ from an earlier version, an additional entry may be presented in the function drop-down menu of the extension manager: UPDATE! This option provides a function to convert some tables of your database. You should read the section below about upgrading to version 2.0+.

Creating a Direct Mail folder

All operations of the Direct Mail module are performed from a specifically configured SysFolder. You may configure multiple Direct Mail folders.

To create a Direct Mail folder:

Create a new page somewhere in the page tree of your site.

Choose it to be of type SysFolder (and nothing else!)

Give it a title

Make sure it contains plugin: Direct Mail.

Once the Direct Mail folder is created, its title should appear in the list of available Direct Mail folders when using backend menu option Web > Direct Mail. Clicking on the folder title should display a page with the module drop-down function menu.

Configuring the Direct Mail module in Page TSConfig

The Direct Mail configuration properties are set in the Page TSConfig of the Direct Mail folder under key mod.web_modules.dmail.

Note that all these properties may conveniently be set using the Direct Mail module function “Module configuration”.

The following properties set default values for corresponding properties of direct mails. These properties of direct mails determine the headers inserted in the direct mail messages.

Property:	Data type:	Description:
from_email	string	Default value for the 'From' or sender email address of direct mails. (Required) Note: This email address appears as the originating address or sender address in the direct mails received by the recipients.
from_name	string	Default value for 'From' or sender name of direct mails. (Required) Note: This name appears as the name of the author or sender in the direct mails received by the recipients.
replyto_email	string	Default value for 'Reply To' email address. Note: This is the email address to which replies to direct mails are sent. If not specified, the 'From' email is used.
replyto_name	string	Default value for 'Reply To' name. Note: This is the name of the 'Reply To' email address. If not specified, the 'From' name is used.
return_path	string	Default return path email address. Note: This is the address to which non-deliverable mails will be returned to. Note: If you put in the marker ###XID###, it'll be substituted with the unique id of the mail recipient. Note: The return path email address cannot be set by the Direct Mail module if PHP is running with safe_mode enabled.
organisation	string	Name of the organization sending the mail.
priority	int+	Default priority of direct mails. Possible values are: 1 - High 3 - Normal 5 – Low Default: 3

The following properties set default values for corresponding properties of direct mails. These properties of direct mails determine the format of the content of direct mail messages.

Property:	Data type:	Description:
sendOptions	int+	Default value for the format of email content. If in doubt, set it to 3 (Plain and HTML). The recipients are normally able to select their preferences anyway. Possible values are: 1 - Plain text only 2 - HTML only 3 - Plain and HTML Default: 3
includeMedia	boolean	Default value for this direct mail option: if set, images and other media are incorporated into the HTML mail content. Note: When this option is set on a direct mail, images and other media are encoded and incorporated into the messages. Sent messages will be heavier to transport. When the option is not set, images and media are included in HTML content by absolute reference (href) to their location on the site where they reside. Default: 0
flowedFormat	boolean	Default value for this direct mail option: if set, text will flow normally in the plain text content of email messages. Note: If the option is set, plain text mail content will still be broken in fixed length lines, as is standard for plain text email content, but so-called flowed format will be used. This will allow client agents that support this format to display the text as normally flowing text. The option is ignored if 'quoted-printable' is used. Note: this setting will produce email headers with 'format=flowed'. See http://www.ietf.org/rfc/rfc3676.txt for more information. Note: In order for plain text content to be correctly rendered for effective use of this option, the flowedFormat property should also be set in the TS template of the plain text rendering plugin.

The following properties set default values for corresponding properties of direct mails. These properties of direct mails specify parameters used to fetch the content of the direct mails.

Property:	Data type:	Description:
HTMLParams	string	Default value for additional URL parameters used to fetch the HTML content from a TYPO3 page. Note: The specified parameters will be added to the URL used to fetch the HTML content of the direct mail from a TYPO3 page. If in doubt, leave it blank.
plainParams	string	Default value for additional URL parameters used to fetch the plain text content from a TYPO3 page. Note: The specified parameters will be added to the URL used to fetch the plain text content of the direct mail from a TYPO3 page. Note: If in doubt, set it either to '&type=99' or, when TemplaVoila is used, to '&print=1'. Default: &type=99
use_domain	Int+	Uid of default sys_domain record of the domain that should be used for fetching content from internal TYPO3 pages. Note: When a domain is specified on a direct mail, the domain will also be used in all internal links contained in mail content. Note: When no domain is specified on a direct mail, the domain in use in the backend when the direct mail is compiled is used to fetch internal TYPO3 pages.

The following properties specify the content transfer encodings and character sets to use when sending mails.

Property:	Data type:	Description:
quick_mail_encoding	string	Content transfer encoding to use when sending quick mails. Possible values: quoted-printable base64 8bit Default: quoted-printable
direct_mail_encoding	string	Default value for the content transfer encoding of direct mails. Possible values: quoted-printable base64 8bit Default: quoted-printable
quick_mail_charset	string	Character set to use when sending quick mails. Default: iso-8859-1
direct_mail_charset	string	Default character set for direct mails built from external pages. Note: This is the character set used in direct mails when they are built from external pages and character set cannot be auto-detected. Note: Direct mails based on internal TYPO3 pages will be sent with the character set in which they are rendered as determined by their TS template. Default: iso-8859-1

The following properties specify how links in mail content are processed.

Property:	Data type:	Description:
use_rdct	boolean	If set, links longer than 76 characters found in plain text content will be redirected: long URL's will be substituted with ?RDCT=[md5hash] parameters. Note: This configuration determines how Quick Mails are handled and further sets the default value for Direct Mails. Default: 0
long_link_mode	boolean	If set and if use_rdct is set, all links in plain text content will be redirected, not only links longer than 76 characters. Default: 0
enable_jump_url	boolean	If set, the use of jump URL's will be enabled so that click statistics can be produced. Default: 0
authcode_fieldList	list	Default list of fields to be used in the computation of the authentication code included in unsubscribe links and in jump URL's in direct mails. Default: uid

The following properties specify parameters for the operations of various functions of the Direct Mail module.

Property:	Data type:	Description:
http_username	string	The username used to fetch the mail content, if mail content is protected by HTTP authentication. Note: The username is NOT sent in the mail! Note: If you do not specify a username and password and a newsletter page happens to be protected, an error will occur and no mail content will be fetched.
http_password	string	The password used to fetch the mail content, if mail content is protected by a HTTP authentication. Note: The password is NOT sent in the mail! Note: If you do not specify a username and password and a newsletter page happens to be protected, an error will occur and no mail content will be fetched.
userTable	string	Custom-defined table that may be used to send direct mails in addition to fe_users and tt_address tables. Note: The following columns must be defined in the custom-defined table: uid, name, title, email, phone, ww, address, company, city, zip, country, fax, module_sys_dmail_category, module_sys_dmail_html
test_tt_address_uids	list of UIDs	List of UID numbers of test recipients. Before sending mails, you should test the mail content by sending test mails to one or more test recipients. The available recipients for testing are determined by this list of UID numbers. So first, find out the UID numbers of the recipients you wish to use for testing, then enter them here in a comma-separated list.
test_dmail_group_uids	list of UIDs	List of UID numbers of test recipient lists. Alternatively to sending test-mails to individuals, you can choose to send to a whole list. This is the list of recipient list UID numbers available for this action.

Configuring the Direct Mail Wizard in User TSConfig

The following properties may be used in the User TSConfig (BE user or BE usergroups) to disable some options in the Direct Mail wizard (send wizard).

Property:	Data type:	Description:
hideTabs	string	Hide the options of the direct mail source in the first step. To hide more than one options, you can separate the values with comma. Available value: int: hide the internal page option ext: hide the external page option quick: hide the quickmail option dmail: hide the direct mail option
hideSteps	string	Only to hide the third step (categorizing content). Available value= cat

[beuser:tx_directmail]

Configuring the Calendar of the Direct Mail Wizard

The calendar used in the fifth step of the Direct Mail send Wizard is based on the Date Selector Library Extension (rlmp_dateselectlib) from Robert Lemke.

The following properties are all children of mod.web_modules.dmail.calConf and it is used in the Page TSConfig of the Direct Mail SysFolder.

Property:	Data type:	Description:	Default:
weekStartsMonday	boolean	If set, calendar's week start on Monday. Otherwise it will start on Sunday	0
weekend	Comma-list of integer	A comma-list of integers which defines which days form a weekend. Possible values range from 0 to 6, where 0 means Sunday, 1 means Monday etc. The default is Saturday and Sunday (6,0)	6,0
allowedYearMin allowedYearMax	integer	If specified, the calendar only accepts dates in the range of these years.	1900 2070
displayWeekNumbers	boolean	If set, the week numbers will be displayed	1
showMethod	string	There are two ways to display the calendar. Attach below the input field or freely positionable. Possible values are: element: the calendar will be shown right below the input field. Absolute: the calendar will be shown at the specified coordination (see showPositionAbsolute)	element
showPositionAbsolute	offset	Use if showMethod is set to absolute. Example: showPositionAbsolute= 100,150	0,0
stylesheet	string	Defines a valid stylesheet relatively to your websites root URL. Examples: stylesheet = fileadmin/yourstylesheet.css stylesheet = EXT:yourextension/res/stylesheet.css	EXT:direct_mail/res/scripts/calendar/calendar-system.css
inputFieldLabel	string	The input label of the button, which launches the calendar	...

Configuring the TS template for newsletter pages

Newsletter pages are just normal pages. Their rendering is configured by the TS template. However, take the following into consideration:

The TS template should not contain frames.

If you insert forms in the newsletter page, you should use the GET method. POST method may not transfer data to the page.

Configuring plain text rendering

It is good practice to always include a plain text version in email messages. Some email clients are not able to present the html content of email messages, but will present the plain text version if available. Users of many email clients may also have the option to set their preference as to the format that they wish to see.

In order to configure the rendering f your newsletter pages in plain text, include static template “Direct Mail Plain Text” on the TS Template applicable to the pages. This static template contains a pre-defined template for plain text content rendering and makes it accessible as page type 99. You may thereafter configure the template to your needs in the template.

Note: In versions previous to version 2.0, plain text rendering was achieved by including static template "plugin.alt.plaintext (99)". This static template may still be used in version 2.0+.

The “Direct Mail Plain Text” static template is as follows:

Constants

plugin.tx_directmail_pi1 {

# cat=plugin.tx_directmail_pi1//; type=string; label= Site url: Enter the url of the site here.

siteUrl = http://www.example.test/

# cat=plugin.tx_directmail_pi1/enable/; type=boolean; label= Use flowed format: The same option should be set on the direct mail.

FlowedFormat = 0

# cat=plugin.tx_directmail_pi1/enable/; type=boolean; label= Double line feeds: Line feeds found in bodytext will be doubled in the plain text version.

doubleLF = 0

# cat=plugin.tx_directmail_pi1//; type=string; label= Split char to remove from graphical headers: Headers built as GIFBUILDER objects may contain split characters. If specified here, they will be removed from headers of type 5 in the plain text version.

removeSplitChar =

}

Setup

plugin.tx_directmail_pi1 = USER

plugin.tx_directmail_pi1.userFunc = tx_directmail_pi1->main

plugin.tx_directmail_pi1 {

siteUrl = {$plugin.tx_directmail_pi1.siteUrl}

flowedFormat = {$plugin.tx_directmail_pi1.flowedFormat}

header.defaultType = 1

header.date = D-m-Y

header.datePrefix = |###HEADER_DATE_PREFIX### |

header.linkPrefix = | ###HEADER_LINK_PREFIX### |

header.1.preLineLen = 76

header.1.postLineLen = 76

header.1.preBlanks=1

header.1.stdWrap.case = upper

header.2 < .header.1

header.2.preLineChar=*

header.2.postLineChar=*

header.3.preBlanks=2

header.3.postBlanks=1

header.3.stdWrap.case = upper

header.4 < .header.1

header.4.preLineChar= =

header.4.postLineChar= =

header.4.preLineBlanks= 1

header.4.postLineBlanks= 1

header.5.removeSplitChar = {$plugin.tx_directmail_pi1.removeSplitChar}

header.5.preBlanks=1

header.5.autonumber=1

header.5.prefix = |: >> |

defaultOutput (

|

[###UNRENDERED_CONTENT### ###CType### ]

|

)

uploads.header = |###UPLOADS_HEADER###|

images.header = |###IMAGES_HEADER###|

images.linkPrefix = | ###IMAGE_LINK_PREFIX### |

images.captionHeader = |###CAPTION_HEADER###|

bulletlist.0.bullet = |* |

bulletlist.1.bullet = |# |

bulletlist.2.bullet = | - |

bulletlist.3.bullet = |> |

bulletlist.3.secondRow = |. |

bulletlist.3.blanks = 1

menu =< tt_content.menu.20

shortcut =< tt_content.shortcut.20

shortcut.0.conf.tt_content =< plugin.tx_directmail_pi1

shortcut.0.tables = tt_content

bodytext.doubleLF = {$plugin.tx_directmail_pi1.doubleLF}

bodytext.stdWrap.parseFunc.tags {

link =< lib.parseFunc_RTE.tags.link

typolist = USER

typolist.userFunc = tx_directmail_pi1->typolist

typolist.siteUrl = {$plugin.tx_directmail_pi1.siteUrl}

typolist.bulletlist =< plugin.tx_directmail_pi1.bulletlist

typohead = USER

typohead.userFunc = tx_directmail_pi1->typohead

typohead.siteUrl = {$plugin.tx_directmail_pi1.siteUrl}

typohead.header =< plugin.tx_directmail_pi1.header

typocode = USER

typocode.userFunc = tx_directmail_pi1->typocode

typocode.siteUrl = {$plugin.tx_directmail_pi1.siteUrl}

}

}

includeLibs.tx_directmail_pi1 = EXT:direct_mail/pi1/class.tx_directmail_pi1.php

tx_directmail_pi1 >

tx_directmail_pi1 = PAGE

tx_directmail_pi1.typeNum=99

tx_directmail_pi1.config {

disableAllHeaderCode = 1

additionalHeaders = Content-type:text/plain

}

tx_directmail_pi1.10 = TEMPLATE

tx_directmail_pi1.10 {

template = FILE

template.file = EXT:direct_mail/pi1/tx_directmail_pi1_plaintext.tmpl

marks.CONTENT < styles.content.get

marks.CONTENT.renderObj = < plugin.tx_directmail_pi1

marks.DATE = TEXT

marks.DATE.data = date:U

marks.DATE.strftime = %e. %B %Y

}

Configuring plain text rendering of News records

If you insert News records in your newsletter page, using the “Insert records” content element type, you may configure the plain text rendering of these News rendering. Simply add static template “Direct Mail News Plain Text” in your TS template. Note that order is important: “Direct Mail News Plain Text” should come after or below “Direct Mail Plain Text”. You may then tailor the template to your specific needs.

The “Direct Mail Plain Text” static template is as follows:

Setup

plugin.tx_directmail_pi1 {

shortcut.0.conf.tt_news =< plugin.tt_news

shortcut.0.conf.tt_news.code = PLAINTEXT

shortcut.0.conf.tt_news.defaultCode = PLAINTEXT

shortcut.0.conf.tt_news.displayCurrentRecord = 1

shortcut.0.conf.tt_news.plainTextConf < plugin.tx_directmail_pi1

shortcut.0.tables = tt_content,tt_news

tt_news_author.defaultType = 3

tt_news_author.date = D-m-Y

tt_news_author.prefix = |###TT_NEWS_AUTHOR_PREFIX### |

tt_news_author.datePrefix = |###TT_NEWS_AUTHOR_DATE_PREFIX### |

tt_news_author.emailPrefix = | ###TT_NEWS_AUTHOR_EMAIL_PREFIX### |

tt_news_author.1.preLineLen = 76

tt_news_author.1.postLineLen = 76

tt_news_author.1.preBlanks=1

tt_news_author.1.stdWrap.case = upper

tt_news_author.2 < .tt_news_author.1

tt_news_author.2.preLineChar=*

tt_news_author.2.postLineChar=*

tt_news_author.3.preBlanks=1

tt_news_author.3.stdWrap.case = upper

tt_news_author.4 < .tt_news_author.1

tt_news_author.4.preLineChar = =

tt_news_author.4.postLineChar = =

tt_news_author.4.preLineBlanks= 1

tt_news_author.4.postLineBlanks= 1

tt_news_short < .bodytext

tt_news_short.header = |###TT_NEWS_SHORT_HEADER### |

tt_news_bodytext < .bodytext

tt_news_bodytext.header = |###TT_NEWS_BODYTEXT_HEADER### |

}

Configuring the use of categories

If you intend to make use of Direct Mail categories, you should do the following.

First, add static template “Direct Mail Content Boundaries” to the TS template of the Direct Mail folder. This will ensure that content boundaries are inserted on the page when it is rendered whenever categories assignments are found on the content elements of the page. The static template ensures that content boundaries will be inserted in both HTML and plain text content. Insertion of boundaries is specified by setting TS setup property:

config.insertDmailerBoundaries = 1

This setting is already included in static template “Direct Mail Content Boundaries” .

Secondly, categories that may be assigned to content elements, to address records, to FE users or to recipient lists are determined by Page TSConfig. Therefore, you should configure the following properties in the Page TsConfig of the Direct Mail folder and, perhaps, of any page whose subtree may contain records of the corresponding types and which may be used in Direct Mail operations:

TCEFORM.tt_content.module_sys_dmail_category.PAGE_TSCONFIG_IDLIST = pid_list

TCEFORM.tt_address.module_sys_dmail_category.PAGE_TSCONFIG_IDLIST = pid_list

TCEFORM.fe_users.module_sys_dmail_category.PAGE_TSCONFIG_IDLIST = pid_list

TCEFORM.sys_dmail_group.select_categories.PAGE_TSCONFIG_IDLIST = pid_list

where pid_list is the list of page id's on which categories may be found that may be assigned to records of the given type.

Finally, when the use of categories is thus configured on a page and its subtree, you may also want the categories assignment field to be displayed in backend forms. This is achieved by setting the following properties in Page TSConfig of the same pages:

TCEFORM.tt_content.module_sys_dmail_category.disabled = 0

TCEFORM.tt_address.module_sys_dmail_category.disabled = 0

TCEFORM.fe_users.module_sys_dmail_category.disabled = 0

TCEFORM.sys_dmail_group.select_categories.disabled = 0

The “Direct Mail Content Boundaries” static template is as follows:

Setup

// Configuring the insertion of dmailer boundaries

includeLibs.tx_directmail_container = EXT:direct_mail/res/scripts/class.tx_directmail_container.php

// In html content

tt_content.stdWrap.postUserFunc = tx_directmail_container->insert_dMailer_boundaries

// In old plaintext content static tenmplate

lib.alt_plaintext.renderObj.userProc < tt_content.stdWrap.postUserFunc

lib.alt_plaintext.renderObj.userProc.useParentCObj = 1

// In new direct mail plain text plugin

plugin.tx_directmail_pi1.userProc < tt_content.stdWrap.postUserFunc

plugin.tx_directmail_pi1.userProc.useParentCObj = 1

// Enable the insertion of content boundaries

config.insertDmailerBoundaries = 1

Enabling click statistics

To enable the click statistics of the Direct Mail module, enable the following checkbox in the "Module configuration” function:

This will register all links in the email messages and count all clicks on them.

Note that form url's are not directed through the jumpurl feature but rather directly to the target page.

A nice trick is to place a little clear-gif image in your HTML template and put the parameter dmailerping=”1” in the tag. This will force the capture function to set the url of this image absolute through the jumpurl registration. This means in other words that when this mail is opened it will be registered. This is an additional feature to the regular feature which registers all links clicked in the mail.

Example:

<img src="typo3conf/ext/direct_mail/res/gfx/dmailerping.gif" width="1" height="1" dmailerping="1" />

Note that the result of the jumpurl setting on the above HTML line is that the src attribute will be replaced by one that refers to the address of a script. Such attribute in HTML content of email messages may be disabled by SPAM filtering software.

Configuring the mailer engine cron task

The mailer engine may be invoked from the Direct Mail backend module. However, if your newsletters are sent to numerous recipients, this extension provides a mailer engine script that may be configured as a cron task on the server. In this example setting, the mailer engine will be invoked every 5 minutes and will check if any mails need to be sent:

when installed as local extension:

*/5 * * * * /home/....../public_html/typo3conf/ext/direct_mail/res/scripts/dmailerd.phpcron> /dev/null

when installed as global extension:

*/5 * * * * /home/....../public_html/typo3/ext/direct_mail/res/scripts/dmailerd.phpcron> /dev/null

Note that the absolute path to the script must be specified.

Note also that dmailerd.phpcron is a shell script and requires the availability of a PHP binary, "/usr/bin/php”. Depending on your server configuration, you may have to edit the first line of the script to refer to the location of the PHP binary.

Using Sendmail deferred delivery mode

The following option applies only when Sendmail is used in Unix/Linux environments.

When sending a large number of emails with the mailer engine, processing delays may be caused by attempts on the part of the mail server to locate the destination domain of each email. As a consequence, timeouts may occur causing failure of the cron ask. Or the cron task may be triggered before the previous instance has terminated. In the latter case, some emails may be sent multiple times.

Such problems may be avoided by using the Sendmail deferred delivery mode. In this mode, each email is simply put on a queue for later processing by Sendmail. This results in much faster response to the cron task.

You may enable the deferred delivery mode by setting the corresponding variable on the installation screen of this extension. The option applies only to emails sent by the mailer engine cron task of the Direct Mail extension.

The following variables must be set in the Sendmail configuration file (values shown here are just examples):

DAEMON=yes

SMQUEUE=10m

QUEUE=30m

These example settings have the following meaning: the email queue will be emptied every 10 minutes. If a problem is encountered while trying to send an email, it will be put on another queue and delivery will be attempted again every 30 minutes.

For more information on Sendmail configuration, see: http://www.sendmail.org/.

Configuring the analysis of returned mails

There are probably many ways to configure analysis of returned mails. We propose an approach based on open Source program fetchmail. For more information about fetchmail, see The fetchmail home page .

Create a mailbox (popbox) for the returned mails, for example: “bounce@pophost.org”. This mailbox should be located on the same machine as the TYPO3 installation.

Use the Module Configuration function of the Direct mail module to configure this same address in the 'Return Path' field in Page TS Config:

fetchmail can read a mailbox and then do something with these mails. We are going to use fetchmail to “pipe” the returned mails to the “returnmail.phpsh” script. fetchmail uses a configuration file that should be outside the web accessible folder. For examble, it may be positionned on the root: /root/.fetchmailrc. ls -l of this file may look like this:

-rwx--x--- 1 root root 208 Jun 20 12:50 /root/.fetchmailrc

Insert the following line in file .fetchmailrc, substituting variables my.pophost.org with the name of your mailserver, and username-of-popbox and password-of-popbox with the name and password of your bounce mailbox:

poll my.pophost.org timeout 40 username "username-of-popbox" password "password-of-popbox" flush mda "/path/to/your/TYPO3/installation/typo3conf/ext/direct_mail/res/scripts/returnmail.phpsh"

Note that the absolute path to the script must be specified. If the extension is installed as a global extension, substitute typo3conf with typo3 in the above path. Make sure that script returnmail.phpsh has sufficient permissions to be run by the server. Note also that returnmail.phpsh is a shell script and requires the availability of a PHP binary, "/usr/bin/php”. Depending on your server configuration, you may have to edit the first line of the script to refer to the location of the PHP binary.

If you have configured multiple Direct Mail folders each with its own return mailbox, you will need a similar line for each mailbox.

Use the command “crontab -e” (as root), or cPanel tool, to add the following cron task. You need only one, even if you have configured multiple folders of Direct Mail. This example setting will run the cron task every 10 minutes:

*/10 * * * * fetchmail> /dev/null

Use the Direct Mail module to send a newsletter to a bouncing address. Use the Statistics function of the module to verify that the bounced mail is accounted for in the displayed statistics.

Configuring SMTP

To send the Newsletter with SMTP, you only have to enable this feature from the extension manager (Ext Manager > Direct Mail).

Before enabling this feature, you have to make sure, that PEAR:mail is installed on your server. To install PEAR:mail you can use one of the following extensions:

PEAR Mail Library (jr_pear_mail)

PEAR Manager (pear)

This feature is kindly provided by Benjamin Mack.

There is a bug in PEAR, please see “Known Problem” Section.

Upgrading to version 2.0+

The two main considerations when upgrading to version 2.0+ from a previous version are:

the new static templates: see sections “Configuring plain text rendering” and “Configuring the use of categories”;

the conversion of categories: in previous versions, categories were defined in the Page TSConfig of the Direct Mail folder. Version 2.0 introduces a new Direct Mail Category database table.

When upgrading to version 2.0+, it is necessary to convert pre-existing categories in Page TSConfig and create corresponding Direct Mail Categories in the Direct Mail folders. If any categories were assigned to Content elements, Addresses , FE Users or Recipient Lists, these assignments also need to be converted to refer to the newly created Direct Mail Categories.

Conversion of categories is NOT revertible. Therefore, it would be prudent to take a backup of the TYPO3 database before this conversion is performed.

When upgrading to version 2.0, an additional option is presented in the function drop-down menu of the Extension Manager: UPDATE! The additional option is presented if a Direct Mail folder already exists and until at least one Direct Mail Category has been created.

If you select the UPDATE! option, pre-existing categories defined in Page TSConfig will be converted into new Direct Mail Categories. The Direct Mail categories will be created in each of the Direct Mail folders for which category definitions may be found in Page TSConfig.

Conversion of categories assignments in Content elements, Addresses, FE Users and Recipient Lists will also be attempted. If the use of categories was not yet configured, this part of the conversion process will fail, but may re-attempted a later time, using the “Categories conversion” function of the Direct Mail module. The next step is thus to configure the use of categories: see the section of this document on this subject.

Note that the conversion of categories assignments is only simulated, until you effectively confirm the conversion. Once confirmed, the records are updated and the conversion cannot be undone without restoring the table with some database backup.

When the use of categories is configured, you may re-attempt the conversion of categories assignments using the “Categories conversion” option of the function menu of the Direct Mail module. The function will report statistics on the conversion being performed, as well as conversion already done. It will also list the records that could not be converted: this exception would normally be due to the fact that the use of categories is not configured for this type of record in the page subtree in which it is found.

Although it may not be undone, categories conversion may be re-attempted as many times as required without causing any harm.

Upgrading to version 2.5.0

To make sure that your cron task and analysis of the returned mail working, you should check the path to the scripts in the cron task. All scripts can be found in the folder res/scripts.

Troubleshooting

Direct mails are not sent

Here is a checklist to work through:

Direct Mail Boundaries should appear in the source of both the HTML and Plain text versions of the page that is used to build the direct mail; otherwise, personalized mails will be empty and will not be sent.

If the direct mail is HTML only, make sure that the recipient addresses or recipient FE users have HTML enabled; no mails will be sent to recipients that do not have HTML enabled.

Make sure there is content for the recipients. Try to include at least one content element that will be sent to all recipients, even if they subscribe to no category. Personalized mails that are empty will not be sent.

Sending a simple test mail will test that sending mail does work.

Sending a test mail to an individual test address or test recipient list configured with the module configuration function will ensure that personalization of messages is working.

Mass mail is only delivered after sending another mail from same MAC OS X server

Problem description:

Invoking a massmail results in getting an admin-mail which says that the procedure has started.

Hours later I didn't got any newsletter, nor an admin-mail which says that sending has stopped.

After invoking a form-mail from another domain on the same server, I instantly get all those messages, i.e. the admin mail and the newsletter. They all got the same timestamp as the first received admin mail.

I send a newsletter to a small group of users in tt_address. There was only one (1) user in this list.

Discussion:

Asking Google brought the hint that there's some authorization problem with sendmail. chmod'ing and chown'ing is supposed to help.

Reference: http://www.entropy.ch/software/macosx/php/welcome_de.html (german)

Solution:

I had to change the group of the "postfix"-directory as described in the reference, i.e.

sudo chmod g-w /

and

sudo chgrp smmsp /var/spool/postfix

Source:

http://bugs.typo3.org/view.php?id=2570

Known Problems

Please see/report problems in the TYPO3 Bugtracker under project tx_directmail.

You may get support in the use of this extension by subscribing to news://news.netfielders.de/typo3.projects.direct-mail.

SMTP / PEAR

There is a bug in PEAR class in sending email to user with special characters in the name field. Example: it won't send email to a user, whose name is test-directmail.

More of this bug please see PEAR Bugtracker

To-Do List

Please see/submit feature request sin the TYPO3 Bugtracker under project tx_directmail.

You may sponsor further development of this extension by contacting the author of this document or any of the contributors mentioned at the beginning of this document.

Change Log

Version:	Changes:
2.0.0	Initial version of this document.
2.0.1	Fix Option includeMedia was not used when fetching direct mail. All mails were sent using absolute links.
2.1.0	Fix getMimeType method until it is fixed in the core. See bugtracker issue 3172. Systematic review of the use of enableFields and deleteClause methods in mod_web_dmail. Use BEfunc instead of pageSelect method. Fix regression of extension constraints in ext_emconf.php.
2.1.1	Correct sql error when sending mails to a recipient list of type 'Plain list'. Rewrite method getMimeType so that it uses the API form t3lib_div.
2.1.2	Fix bugtracker issue 3298: Fatal error: Call to a member function on a non-object in statistics function. Fix bugtracker issue 3301: tca.php has short php open tag. Remove most field exclusions in tca.php. Issue error messages when content cannot be fetched. Issue warning message when fetched content contains no direct mail boundaries. Include all content in messages when there are no boundaries in the content.
2.1.4	Fix bugtracker issue 3444: missing value-check in tt_news plain-text rendering script. Fix bugtracker issue 3317: too short require_once statements in mod/class.mod_web_dmail.php. Remove references to $HTTP_*_VARS in dmailerd.phpcron and returnmail.phpsh. Fix issue 3489: setting tt_news to "strict" will cause an error in direct mail. Update to the structure of the manual.
2.1.5	Delete file pi1/locallang.php Add Spanish labels in pi1/locallang.xml Replace br tag by linefeed (instead of space) in plain text rendering plugin. Correction: missing character set conversion of mail subject when mail character set is different from the backend charset. Correction: mail subject would be encoded multiple times. Thanks to David Bocher. Chapter "Personalizing direct mail content" was augmented. Yhanks to Thorsten Kahler. Correction: display of some statistics was erroneous. Thanks to David Bocher. Two new constants in the “Direct Mail Plain text” static template: doubleLF and removeSplitChar. Partial fix for bugtracker issue 3207: Recipient list from Website User Group and users with multiple user groups. Will work at least with MySQL.
2.5.0	Wizard based sending. Wizard based importing of csv records. Main functions (Direct Mail, Recipient Lists, Statistics, Mailer Engine, Configuration) in separate modules.

EXT: Direct Mail - 26

EXT: Direct Mail

Kasper Skårhøj

2002-11-01T00:32:00

Ivan Dharma Kartolo

Kasper Skårhøj