Eset Mail Security 4 for Microsoft Exchange Server Version 4.2 Installation Manual and User Guide

ESET Mail Security 4

for Microsoft Exchange Server

Version 4.2

Installation Manual and User Guide

Microsoft® Windows® Server 2000 / 2003 / 2008

Contents

ESET Mail Security

ESET Mail Security was developed by ESET, spol. s r.o. For more information visit www.eset.com. All rights reserved. No part of this documentation may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise without permission in writing from the author. ESET, spol. s r.o. reserves the right to change any of the described application software without prior notice.

1. Introduction

2. Installation

3. Update

................................................3

........................................................................3System requirements1.1

........................................................................3Methods used1.2

..............................................................................3Mailbox scanning via VSAPI1.2.1

..............................................................................4Message filtering on the SMTP server level1.2.2

........................................................................4Types of protection1.3

..............................................................................4Antivirus protection1.3.1

..............................................................................4Antispam protection1.3.2

..............................................................................4Application of user-defined rules1.3.3

................................................5

........................................................................5Typical Installation2.1

........................................................................7Custom Installation2.2

........................................................................11Upgrading to a newer version2.3

........................................................................11Installation in a clustered environment2.4

........................................................................13License2.5

........................................................................14Post-Installation Configuration2.6

................................................16

........................................................................17Proxy server setup3.1

4. ESET Mail Security - Microsoft

................................................19

Exchange Server protection

........................................................................19General settings4.1

..............................................................................19Rules4.1.1

..............................................................................20Adding new rules4.1.1.1

..............................................................................20Actions4.1.1.2

..............................................................................22Log files4.1.2

..............................................................................23Message quarantine4.1.3

..............................................................................23Adding a new quarantine rule4.1.3.1

..............................................................................24Performance4.1.4

..............................................................................25Transport Agent4.1.5

........................................................................26Antivirus and antispyware settings4.2

..............................................................................26Actions4.2.1

..............................................................................27Alerts and notifications4.2.2

..............................................................................28Performance4.2.3

4.2.4

4.2.4.4

........................................................................40Antispam settings4.3

........................................................................46FAQ4.4

Virus-Scanning Application Programming

..............................................................................28

Interface (VSAPI)

..............................................................................29Microsoft Exchange Server 5.5 (VSAPI 1.0)4.2.4.1

.............................................................................30Actions4.2.4.1.1

.............................................................................31Performance4.2.4.1.2

..............................................................................31Microsoft Exchange Server 2000 (VSAPI 2.0)4.2.4.2

.............................................................................32Actions4.2.4.2.1

.............................................................................33Performance4.2.4.2.2

..............................................................................34Microsoft Exchange Server 2003 (VSAPI 2.5)4.2.4.3

.............................................................................35Actions4.2.4.3.1

.............................................................................35Performance4.2.4.3.2

Microsoft Exchange Server 2007/2010 (VSAPI

..............................................................................36

2.6)

.............................................................................37Actions4.2.4.4.1

.............................................................................38Performance4.2.4.4.2

..............................................................................39Transport Agent4.2.5

..............................................................................40Antispam engine parameter setup4.3.1

..............................................................................41Configuration file4.3.1.1

..............................................................................45Alerts and notifications4.3.2

..............................................................................45Transport Agent4.3.3

Customer Care Worldwide: www.eset.eu/support Customer Care North America: www.eset.com/support

REV. 11/2/2010

1. Introduction

ESET Mail Security 4 for Microsoft Exchange Server is an integrated solution protecting user mailboxes from various types of malware content (most often they are email attachments infected by worms or trojans, documents containing harmful scripts, phishing, spam etc.). ESET Mail Security provides three types of protection: Antivirus, Antispam and application of user-defined rules. ESET Mail Security filters the malicious content on the mailserver level, before it arrives in the addressee’s email client inbox.

ESET Mail Security supports Microsoft Exchange Server versions 5.5 and later as well as Microsoft Exchange Server in a cluster environment. In newer versions (Microsoft Exchange Server 2007 and later), specific roles (mailbox, hub, edge) are also supported. You can remotely manage ESET Mail Security in larger networks with the help of ESET Remote Administrator.

As far as functionality is concerned, ESET Mail Security is very similar to ESET NOD32 Antivirus 4.0. It has all the tools necessary to ensure protection of the server-as-client (resident protection, web-access protection, email client protection and antispam), while providing Microsoft Exchange Server protection.

1.1 System requirements

Supported Operating Systems:

Microsoft Windows Server 2000 Microsoft Windows Server 2003 (x86 and x64) Microsoft Windows Server 2008 (x86 and x64) Microsoft Windows Server 2008 R2 Microsoft Windows Small Business Server 2003 Microsoft Windows Small Business Server 2003 R2 Microsoft Windows Small Business Server 2008

Supported Microsoft Exchange Server versions:

Microsoft Exchange Server 5.5 SP3, SP4 Microsoft Exchange Server 2000 SP1, SP2, SP3 Microsoft Exchange Server 2003 SP1, SP2 Microsoft Exchange Server 2007 SP1, SP2 Microsoft Exchange Server 2010 SP1

Hardware requirements depend on the operating system version and the version of Microsoft Exchange Server in use. We recommend reading the Microsoft Exchange Server product documentation for more detailed information on hardware requirements.

1.2 Methods used

Two independent methods are used to scan email messages:

Mailbox scanning via VSAPI Message filtering on the SMTP server level

1.2.1 Mailbox scanning via VSAPI

The mailbox scanning process is triggered and controlled by the Microsoft Exchange Server. Emails in the Microsoft Exchange Server store database are scanned continuously. Depending on the version of Microsoft Exchange Server, the VSAPI interface version and the user-defined settings, the scanning process can be triggered in any of the following situations:

When the user accesses email, e.g. in an email client (email is always scanned with the latest virus signature database)

In the background, when use of the Microsoft Exchange Server is low

Proactively (based on the Microsoft Exchange Server’s inner algorithm)

The VSAPI interface is currently used for antivirus scan and rule-based protection.

1.2.2 Message filtering on the SMTP server level

SMTP server-level filtering is secured by a specialized plugin. In Microsoft Exchange Server 2000 and 2003, the plugin in question (Event Sink) is registered on the SMTP server as a part of Internet Information Services (IIS). In Microsoft Exchange Server 2007/2010, the plugin is registered as a transport agent on the Edge or the Hub roles of the Microsoft Exchange Server.

SMTP server-level filtering by a transport agent provides protection in the form of antivirus, antispam and userdefined rules. As opposed to VSAPI filtering, the SMTP server-level filtering is performed before the scanned email arrives in the Microsoft Exchange Server mailbox.

1.3 Types of protection

There are three types of protection:

1.3.1 Antivirus protection

Antivirus protection is one of the basic functions of the ESET Mail Security product. It guards against malicious system attacks by controlling file, email and Internet communication. If a threat with malicious code is detected, the Antivirus module can eliminate it by first blocking it and then cleaning, deleting or moving it to quarantine.

1.3.2 Antispam protection

Antispam protection integrates several technologies (RBL, DNSBL, Fingerprinting, Reputation checking, Content analysis, Bayesian filtering, Rules, Manual whitelisting/blacklisting, etc.) to achieve maximum detection of email threats. The antispam scanning engine’s output is the spam probability value of the given email message expressed as a percentage (0 to 100). Values of 90 and above are considered sufficient for ESET Mail Security to classify an email as spam.

Another component of the antispam protection module is the Greylisting technique (disabled by default). The technique relies on the RFC 821 specification, which states that since SMTP is considered an unreliable transport, every message transfer agent (MTA) should repeatedly attempt to deliver an email after encountering a temporary delivery failure. A substantial part of spam consists of one-time deliveries (using specialized tools) to a bulk list of email addresses generated automatically. A server employing Greylisting calculates a control value (hash) for the envelope sender address, the envelope recipient address and the IP address of the sending MTA. If the server cannot find the control value for the triplet within its own database, it refuses to accept the message, returning a temporary failure code (temporary failure, for example, 451). A legitimate server will attempt a redelivery of the message after a variable time period. The triplet’s control value will be stored in the database of verified connections on the second attempt, allowing any email with relevant characteristics to be delivered from then on.

1.3.3 Application of user-defined rules

Protection based on user-defined rules is available for scanning with both the VSAPI and the transport agent. You can use the ESET Mail Security user interface to create individual rules that may also be combined. If one rule uses multiple conditions, the conditions will be linked using the logical operator AND. Consequently, the rule will be executed only if all its conditions are fullfilled. If multiple rules are created, the logical operator OR will be applied, meaning the program will run the first rule for which the conditions are met.

In the scanning sequence, the first technique used is greylisting - if it is enabled. Consequent procedures will always execute the following techniques: protection based on user-defined rules, followed by an antivirus scan and, lastly, an antispam scan.

2. Installation

After purchase, the ESET Mail Security installer can be downloaded from ESET’s website as an .msi package. Once you launch the installer, the installation wizard will guide you through the basic setup. There are two types of installation available with different levels of setup details:

1. Typical Installation

2. Custom Installation

2.1 Typical Installation

Typical installation provides configuration options appropriate for most users. The settings provide excellent security coupled with ease of use and high system performance. Typical installation is the default option and is recommended if you do not have the particular requirements for specific settings.

After selecting the installation mode and clicking Next, you will be prompted to enter your username and password for automatic updates of the program. This plays a significant role in providing constant protection of your system.

Enter your Username and Password, i.e., the authentication data you received after the purchase or registration of the product, into the corresponding fields. If you do not currently have your username and password available, authentication data can be inserted at any time, directly from the program.

In the next step - License Manager - Add the license file delivered via email after product purchase.

The next step is configuration of the ThreatSense.Net Early Warning System. The ThreatSense.Net Early Warning System helps ensure that ESET is immediately and continuously informed about new infiltrations in order to quickly protect its customers. The system allows for submission of new threats to ESET‘s Threat Lab, where they are analyzed, processed and added to the virus signature database.

By default, the Enable ThreatSense.Net Early Warning System option is selected, which will activate this feature. Click Advanced setup... to modify detailed settings for the submission of suspicious files.

The next step in the installation process is to configure Detection of potentially unwanted applications. Potentially unwanted applications are not necessarily malicious, but can often negatively affect the behavior of your operating system.

These applications are often bundled with other programs and may be difficult to notice during the installation process. Although these applications usually display a notification during installation, they can easily be installed without your consent.

Select the Enable detection of potentially unwanted applications option to allow ESET Mail Security to detect this type of threat (recommended). The final step in Typical installation mode is to confirm installation by clicking the Install button.

2.2 Custom Installation

Custom installation is designed for users who have experience with fine-tuning programs and who wish to modify advanced settings during installation.

After selecting the installation mode and clicking Next, you will be prompted to select a destination location for the installation. By default, the program installs in C:\Program Files\ESET\ESET Mail Security\. Click Browse… to change this location (not recommended).

Next, Enter your Username and Password. This step is the same as in Typical installation (see “Typical installation” ).

In the next step - License Manager - add the license file delivered via email after product purchase.

After entering your username and password, click Next to proceed to Configure your Internet connection.

If you use a proxy server, it must be correctly configured for virus signature updates to work correctly. If you do not know whether you use a proxy server to connect to the Internet, select the default setting I am unsure if my Internet connection uses a proxy server. Use the same settings as Internet Explorer (Recommended) and click Next. If you do not use a proxy server, select the I do not use a proxy server option.

To configure your proxy server settings, select I use a proxy server and click Next. Enter the IP address or URL of your proxy server in the Address field. In the Port field, specify the port where the proxy server accepts connections (3128 by default). In the event that the proxy server requires authentication, enter a valid Username and Password to grant access to the proxy server. Proxy server settings can also be copied from Internet Explorer if desired. To do this, click Apply and confirm the selection.

Click Next to proceed to Configure automatic update settings. This step allows you to designate how automatic program component updates will be handled on your system. Click Change... to access the advanced settings.

If you do not want program components to be updated, select the Never update program components option. Select the Ask before downloading program components option to display a confirmation window before downloading program components. To download program component upgrades automatically, select the Always

update program components option.

NOTE: After a program component update, a restart is usually required. We recommend selecting the If necessary, restart computer without notifying option.

The next installation window offers the option to set a password to protect your program settings. Select the Protect configuration settings with a password option and choose a password to enter in the New password and Confirm new password fields.

The next two installation steps, ThreatSense.Net Early Warning System and Detection of potentially unwanted applications are the same as in Typical installation (see “Typical installation” ).

Click Install in the Ready to install window to complete installation.

2.3 Upgrading to a newer version

Newer versions of ESET Mail Security are issued to bring improvements or fix issues that cannot be remedied by automatic update of the program modules. Upgrade to a newer version can be accomplished in several ways:

Automatically by means of a program component update (PCU)

1. Since program component updates are distributed to all users and may have impact on certain system configurations, they are issued after a long period of testing for seamless upgrade on all possible system configurations. If you need to upgrade to a newer version instantly after it has been released, use either of the following methods.

Manually by downloading and installing a new version over the previous one

2. At the beginning of installation, you can choose to preserve current program settings by selecting the Use current settings check box

Manually with automatic deployment in a network environment by means of ESET Remote Administrator

2.4 Installation in a clustered environment

A cluster is a group of servers (a server connected to a cluster is called a "node") that work together as a single server. This type of environment provides high accessibility and reliability of available services. If one of the nodes in the cluster fails or becomes inaccessible, its functioning is automatically covered by another node in the cluster. ESET Mail Security fully supports Microsoft Exchange Servers connected in a cluster. In order for ESET Mail Security to function properly, it is important that each node in a cluster contains the same configuration. This can be achieved by applying a policy using ESET Remote Administrator (ERA). In the following chapters we will describe how to install and configure ESET Mail Security on servers in a clustered environment using ERA.

Installation

This chapter explains the push installation method; however this is not the only way to install a product on the target computer. For information on additional installation methods, refer to the ESET Remote Administrator User Guide.

1) Download the ESET Mail Security msi installation package from the ESET website to the computer where ERA is installed. In ERA > Remote Install tab > Computers, right-click on any computer from the list and choose Manage

Packages from the context menu. In the Type drop-down menu, select ESET Security Products package and click Add... In the Source, locate the downloaded ESET Mail Security installation package and click Create.

2) In Edit/Select configuration associated with this package, click Edit and configure the settings of ESET Mail Security according to your needs. ESET Mail Security settings are in the following branches: ESET Smart Security,

ESET NOD32 Antivirus > Mail server protection and Mail server protection for Microsoft Exchange Server. You may also set the parameters of other modules included in ESET Mail Security (e.g., Update module, Computer scan, etc.). We recommend exporting configured settings into an xml file which you can later use, e.g. when creating installation package, applying Configuration Task or a Policy.

3) Click Close. In the next dialog window (Do you want to save the package into server?) select Yes and type the name of the installation package. The finished installation package (including name and configuration) will be saved on the server. Most frequently, this package is used for a Push Installation, but it is also possible to save it as a standard

msi installation package and use it for a direct installation on the server (in the Installation Packages Editor > Save As...).

4) Now that the installation package is ready, you can initiate the remote installation on the nodes within a cluster. In the ERA > Remote Install tab > Computers, select the nodes on which you want to install ESET Mail Security (Ctrl + Left-click or Shift + Left-click). Right-click on any of selected computers and select Push Installation from the context menu. Using the Set / Set All buttons, set the Username and Password of a user on the target computer (this must be a user with administrator rights). Click Next to choose the installation package and initiate the remote installation process by clicking Finish. The installation package containing ESET Mail Security and custom configuration settings will be installed on selected target computers/nodes. After a short time, clients with ESET Mail Security will appear in the ERA > Clients tab. You may now manage the clients remotely.

NOTE: For a seamless remote installation process, it is necessary to fulfill certain conditions on the target computers as well as on the ERA Server. For further details, refer to the ESET Remote Administrator User Guide.

Configuration

For ESET Mail Security to function correctly on the nodes within a cluster, the nodes must have the same configuration at all times. This condition is met if you followed the push installation method above. However, there is a chance that the configuration will be changed by mistake, causing inconsistencies between ESET Mail Security products within a cluster. You can avoid this by using a policy in ERA. A policy is very similar to a standard Configuration Task – it sends the configuration defined in the Configuration Editor to the client(s). A policy is different from a Configuration Task because it is continuously applied to the client(s). So the Policy can be defined as a configuration that is regularly forced to a client / group of clients.

In ERA > Tools > Policy Manager... there is a number of options on how to use a policy. The easiest option is to use Default Parent Policy which also generally serves as Default policy for primary clients. This kind of policy is automatically applied to all currently connected clients (in this case, to all ESET Mail Security products within a cluster). You can configure the Policy by clicking Edit..., or use existing configuration saved in the xml file, if you have already created one.

The second option is to create a new policy (Add New Child Policy) and use the Add Clients... option to assign all ESET Mail Security products to this policy.

This configuration ensures a single policy with the same settings will be applied to all clients. If you wish to modify existing settings of an ESET Mail Security server within a cluster, it is sufficient to edit the current policy. Changes will be applied to all clients assigned to this policy.

NOTE: Refer to the ESET Remote Administrator User Guide for detailed information on policies.

2.5 License

A very important step is to enter the license file for ESET Mail Security for Microsoft Exchange Server. Without it, email protection on the Microsoft Exchange Server will not work properly. If you do not add the license file during installation, you can do so later in the advanced settings, under Miscellaneous > Licenses.

ESET Mail Security 4 for Microsoft Exchange Server (EMSX) compares the number of mailboxes for the active directory to your license count. Each Exchange server's active directory is checked to determine the total mailbox count. There is no way to determine which mailboxes are protected and which ones are excluded from protection. Resource mailboxes (i.e. a conference room mailbox) will be tallied in the mailbox count. Email aliases are not tallied in the mailbox count. In a clustered environment, nodes with clustered mailbox role are not tallied in the mailbox count.

To determine how many Exchange enabled mailboxes you have, open Active Directory users and computers on the

server. Right-click on the domain and click Find.... Then from the Find drop-down menu select Custom search and

click the Advanced tab. Paste in the following Lightweight Directory Access Protocol (LDAP) query and click Find Now :

(&(objectClass=user)(objectCategory=person)(mailNickname=*)(|(homeMDB=*) (msExchHomeServerName=*))(!(name=SystemMailbox{*))(!(name=CAS_{*))(msExchUserAccountControl=0) (!userAccountControl:1.2.840.113556.1.4.803:=2))

If the number of mailboxes in your active directory exceeds your license count a message will be entered into your Microsoft Exchange Server log reading, "Protection status changed due to exceeded number of mailboxes (count) covered by your license (count)." Your ESET Mail Security will also notify you by changing its Protection status to

ORANGE and displaying a message informing you that you have 42 days left before your protection will be disabled. If

you receive this notification, please contact your sales representative to purchase additional licenses.

If the 42 days period has passed and you did not add the required licenses to cover the exceeding mailboxes, your Protection status will change to RED. The message will inform you that your protection has been disabled. If you receive this notification, immediately contact your sales representative to purchase additional licenses.

2.6 Post-Installation Configuration

There are several options that have to be configured after the product installation.

Antispam protection setup

This section describes the settings, methods and techniques you can use to protect your network from spam. We recommend reading the following instructions carefully before choosing the most suitable combination of settings for your network.

Spam management

To ensure a high level of Antispam protection you must set actions to be performed on messages already marked as SPAM.

There are three options available:

Deleting spam

The criteria for a message to be marked as SPAM by ESET Mail Security are set reasonably high, decreasing the chances of deleting legitimate email. The more specific the Antispam settings, the less likely it is to delete legitimate email. Advantages of this method include very low consumption of system resources and less administration. The drawback to this method is that if a legitimate email is deleted it cannot be restored locally.

Quarantine

This option excludes the risk of deleting legitimate email. Messages can be restored and resent to the original recipients immediately. The drawbacks of this method are higher consumption of system resources and additional time required for email quarantine maintenance. You can use two methods to quarantine email:

A. Internal Exchange Server quarantine:

- If you want to use the internal server quarantine make sure the Common message quarantine field on the right pane in the advanced settings menu (under Mail server protection > Message quarantine) is left blank. Also make sure that the Quarantine message to the mail server system quarantine option is selected from the drop-down menu at the bottom.

B. Custom quarantine mailbox:

- If you type the desired mailbox in the Common message quarantine field ESET Mail Security will move all new spam messages into your custom mailbox.

Forwarding spam

Spam will be forwarded along to its recipient. However, ESET Mail Security will fill in the relevant MIME header with the SCL value into each message. Based on the SCL value the relevant action will be executed by the Exchange server IMF (Intelligent Message Filtering).

Spam filtering

Antispam Engine

The Antispam engine offers the three following configurations - Recommended, Most accurate, Fastest. If there is no need to optimize your configuration to allow maximum throughput (e.g. high server load), we recommend you select the Most accurate option. When the Recommended configuration is set, the server will automatically adjust its settings based on scanned messages to balance the load. When Most accurate is enabled, the

settings will be optimized in regard to the catch rate. Clicking Custom > Open configuration file allows a user to edit the spamcatcher.conf file. This option is recommended for advanced users only.

Before starting full operation, we recommend that you manually configure the lists of restricted and allowed IP addresses. To do so:

Open the Advanced settings window and navigate to the section Antispam protection > Mail server protection.

Make sure to check the Enable mail server antispam protection field.

Click the Setup... button to set Allowed, Ignored and Blocked IP addresses lists.

The Blocked IP addresses tab contains the list of restricted IP addresses, i.e., if any non-ignored IP in Received headers matches the address on this list, the message scores 100 and no other checks are made. The Allowed IP addresses tab lists all IP addresses that are approved, i.e., if the first non-ignored IP in Received headers matches any address on this list, the message scores 0 and no other checks are made. The Ignored IP addresses tab lists addresses that should be ignored during Real-time Blackhole List (RBL) checks. The list should include all internal IP addresses in the firewall not directly accessible from the Internet. Doing so prevents unnecessary checks and helps to differentiate the external connecting IP addresses from the internal IP addresses.

Greylisting

Greylisting is a method protecting users from spam using the following technique: Transport agent sends a “temporarily reject” SMTP return value (default is 451/4.7.1) for any email from a sender it does not recognize. A legitimate server will attempt to redeliver the message. Spammers typically do not attempt to redeliver messages, because they go through thousands of email addresses at a time and typically cannot spend extra time on resending.

When evaluating the message source, the method takes into account the configurations of the Approved IP addresses list, the Ignored IP addresses list, the Safe Senders and Allow IP lists on the Exchange server and the AntispamBypass settings for the recipient mailbox. Greylisting must be thoroughly configured, or else unwanted operational flaws (e.g. delays in legitimate message deliveries etc.) may occur. These negative effects recede continuously as this method fills the internal whitelist with trusted connections. If you are not familiar with this method, or if you consider its negative side-effect unacceptable, we recommend that you disable the method in the Advanced settings menu under Antispam protection > Mail server protection > Microsoft Exchange Server >

Transport agent > Enable Greylisting.

We recommend disabling greylisting if you intend to test the product's basic functionalities and do not want to configure the advanced features of the program.

NOTE: Greylisting is an additional layer of antispam protection and does not have any effect on the spam evaluation capabilities of the antispam module.

Antivirus protection setup

Quarantine

Depending on the type of cleaning mode you are using we recommend that you configure an action to be performed on infected (not cleaned) messages. This option can be set in the Advanced settings window > Antivirus and

antispyware > Mail server protection > Microsoft Exchange Server > Transport agent.

If the option to move messages into email quarantine is enabled, you need to configure the quarantine under the section Message quarantine in the Advanced settings window.

Performance

If there are no other restrictions, our recommendation is to increase the number of ThreatSense scan engines according to this formula: number of scan engines = number of scan threads and increase the number of VSAPI scanning threads based on this formula: number of scan threads = (number of physical CPUs * 2) + 1 in the Advanced settings window under Antivirus and antispyware > Performance.

NOTE: We recommend that you set the number of ThreatSense scan engines equal to the number of scan threads used.

+ 33 hidden pages