HP TippingPoint Next Generation Firewall Getting Started Guide

HP TippingPoint vSMS Getting Started

Version 4.0

Abstract

This information describes the installation and configuration of the HP TippingPoint Virtual Security Management System (vSMS). This information is for system administrators, technicians, and maintenance personnel responsible for installing, configuring, and maintaining HP TippingPoint vSMS appliances.

*5998-5019*

Part number: 5998-5019 First edition: August 2013

Legal and notice information

Hewlett-Packard Company makes no warranty of any kind with regard to this material, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall not be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance, or use of this material.

This document contains proprietary information, which is protected by copyright. No part of this document may be photocopied, reproduced, or translated into another language without the prior written consent of Hewlett-Packard. The information is provided “as is” without warranty of any kind and is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

TippingPoint®, the TippingPoint logo, and Digital Vaccine® are registered trademarks of Hewlett-Packard All other company and product names may be trademarks of their respective holders. All rights reserved. This document contains confidential information, trade secrets or both, which are the property of Hewlett-Packard No part of this documentation may be reproduced in any form or by any means or used to make any derivative work (such as translation, transformation, or adaptation) without written permission from Hewlett-Packard or one of its subsidiaries.

HP TippingPoint vSMS Getting Started

Publication Part Number: 5998-5019

Product Part Number: N/A

Table of Contents

About This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Target Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Related Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Document Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Typefaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Document Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Customer Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x

Contact Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x

1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

vSMS System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Installation Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Installing SMS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Where to Go Next . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

vSMS Getting Started i

List of Procedures

About This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

How To: Validate the VMware Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

How To: Obtain the vSMS Software from the TMC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

How To: Obtain the vSMS Certification String . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

How To: Deploy the vSMS Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

How To: Start the vSMS Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

How To: Configure the SMS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

How To: Install the Client Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

How To: Complete Initial Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

vSMS Getting Started iii

List of Tables

1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Table 1-1 - VMware vSphere Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Table 1-2 - Installation Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

vSMS Getting Started v

About This Guide

The vSMS Getting Started provides instructions for installing the Virtual Security Management System (vSMS) software-based SMS appliance in a VMware environment. This guide includes the following information:

• Overview, page 1, describes system requirements, provides migration information, and gives a summary of the installation process.

• Installation, page 3, provides detailed instructions for installing and configuring vSMS.

This section covers the following topics:

• Target Audience, page vii

• Related Documentation, page vii

• Document Conventions, page viii

• Customer Support, page x

Target Audience

This guide is intended for security network administrators and specialists that have the responsibility of monitoring, managing, and improving system security. The audience for this material is expected to be familiar with vSMS and VMware and have the following knowledge and skills:

•TCP/IP

•UDP

•ICMP

•Ethernet

•SNTP

•SMTP

•SNMP

Document Conventions

This guide uses the following document conventions.

• Typefaces, page viii

• Document Messages, page ix

Typefaces

HP TippingPoint publications use the following typographic conventions for structuring information:

Convention Element

Medium blue text

Bold font

Italics font

Monospace font

Monospace, italic font

Monospace, bold font

• Cross-reference links

• Email addresses

•Key names

• UI elements that are clicked or selected. Example: Click OK to

accept.

• Text emphasis

•Variables

• Publication titles

• File and directory names

•System output

•Code

• Text typed at the command-line

• Code variables

• Command-line variables

• Emphasis on:

• File and directory names

•System output

•Code

• Text typed at the command line

viii

Document Messages

Document messages are special text that is emphasized by font, format, and icons. This <manual type> contains the following types of messages:

• Warning

•Caution

•Note

•Tip

WARNING! Warning notes alert you to potential danger of bodily harm or other potential harmful

consequences.

CAUTION: Caution notes provide information to help minimize risk, for example, when a failure to follow

directions could result in damage to equipment or loss of data.

NOTE: Notes provide additional information to explain a concept or complete a task. Notes of specific

importance in clarifying information or instructions are denoted as such.

IMPORTANT: Another type of note that provides clarifying information or specific instructions.

TIP: Tips provide helpful hints and shortcuts, such as suggestions about how you can perform a task more

easily or more efficiently.

vSMS Getting Started ix

Customer Support

HP TippingPoint is committed to providing quality customer support to all customers. Each customer receives a customized support agreement that provides detailed support contact information. When you need technical support, refer to your support agreement or use the following information to contact Customer Support.

Before You Contact Support

For a quick and efficient resolution of your problem, take a moment to gather some basic information from before you contact HP TippingPoint customer support:

Information Find It Here...

Your customer number Customer Support Agreement or the shipping invoice that came

with the appliance

SMS serial number Bottom of the SMS server chassis, or use SMS CLI

SMS version number In the SMS client, on the Admin screen, or in the Updates area of

TOS version number In the SMS client, on the Devices screen (an entry for each

DV Toolkit version number In the SMS client, on the Profiles (DV Toolkit Packages) screen

Managed device serial numbers Local Security Manager Dashboard or the shipping invoice that

Contact Information

For additional information or assistance, contact the HP Networking Support:

http://www.hp.com/networking/support

Before contacting HP, collect the following information:

• Product model names and numbers

• Technical support registration number (if applicable)

• Product serial numbers

• Error messages

• Operating system type and revision level

• Detailed questions

key command.

the SMS dashboard

device)

came with the appliance

Contact an HP Authorized Reseller

For the name of the nearest HP authorized reseller, see the contact HP worldwide Web site:

http://www.hp.com/country/us/en/wwcontact.html

1Overview

The Virtual Security Management System (vSMS) is a software-based SMS appliance that operates within a VMware virtual environment. You must have a supported VMware environment installed and configured before you install the vSMS.

With very few exceptions, the vSMS provides the same functionality, the same user interfaces, and operates the same as a physical SMS appliance. Before you install the vSMS, see the latest HP TippingPoint SMS Release Notes available at http://www.hp.com/support/manuals vSMS, use the SMS documentation to operate and administer the vSMS.

This section has the following topics:

• vSMS System Requirements, page 1

• Migration, page 2

• Installation Summary, page 2

vSMS System Requirements

The vSMS uses a VMware Open Virtualization Format (OVF) file to operate. OVF is a packaging and distribution format for virtual machines. You must use the VMware vCenter Server to deploy the .ovf file.

VMware vSphere Environment

. After you install the

To install and use the vSMS, you must have the following VMware vSphere environment set up. Table 1-1 VMware vSphere Environment

Product Version

VMware vCenter Server 5.0 or later

VMware vSphere Client 5.0 or later

VMware ESX/ESXi 5.0 or later

NOTE: Use the vCenter Server to deploy the .ovf file; deploying the file directly through ESX/ESXi utilities

is not supported.

Minimum System Requirements

The vSMS must meet the following minimum system requirements:

• 300 GB virtual disk size for new installation, 73 GB virtual disk size for migration from a previous version

•2 virtual CPUs

• 6 GB memory

• 2 virtual network adapters

Do Not Change the vSMS vNIC Settings

The virtual network interface controller (vNIC) settings configured during deployment of the vSMS are required for the vSMS to operate successfully. For best results, do not change the vNIC settings.

vSMS Getting Started 1

Migration

Automatic migration from vSMS v3.2 to vSMS 4.0 is not supported. You must redeploy the vSMS to migrate. For vSMS v3.3, you can perform an incremental upgrade to vSMS v4.0 without redeploying the vSMS.

To migrate from v3.2 to 4.0, redeploy the vSMS as follows:

1. Back up the vSMS v3.2 database.

2. Remove the vSMS v3.2 virtual appliance from the VMware environment.

3. Deploy the vSMS 4.0 virtual appliance into the VMware environment.

4. Restore the backed up v3.2 database instance.

NOTE: Alternatively, if you have sufficient resources on your ESX/ESXi host, you can shutdown the vSMS

v3.2 virtual appliance, turn it off, deploy the vSMS v4.0 virtual appliance, and restore the backed up database. After you verify the integrity of the restored database instance, you can then delete the old v3.2 virtual appliance from the VMware environment.

NOTE: For added assurance, use vSphere to take a snapshot of the vSMS v3.2 virtual appliance.

Installation Summary

The HP TippingPoint vSMS installation and configuration involves the following components:

• VMware vCenter Server

• VMware vSphere Client

• VMware ESX/ESXi

• vSMS software package, consisting of the vSMS VMware Open Virtualization Format (OVF) file and a .vmdk file

• vSMS Certification String

• vSMS software package MD5 checksum

To install the vSMS, validate the VMware environment where you want to deploy the vSMS, obtain the vSMS and MD5 checksum from the TMC, obtain the vSMS certification string from HP TippingPoint, and then perform the deployment using the following steps.

Table 1-2 Installation Summary

Step 1 Validate the VMware Environment, page 3

Step 2 Obtain the vSMS Software from the TMC, page 3

Step 3 Obtain the vSMS Certification String, page 4

Step 4 Deploy the vSMS Software, page 4

Step 5 Start the vSMS Software, page 5

Step 6 Configure the SMS Server, page 5

Step 7 Install the Client Software, page 6

Step 8 Complete Initial Setup, page 6

2Overview

2Installation

This section provides instructions for installing vSMS in a VMware environment. This section has the following topics:

• Before You Begin, page 3

• Installing SMS, page 3

• Where to Go Next, page 7

Before You Begin

See ”Installation Summary” on page 2 and the latest HP TippingPoint SMS Release Notes available on the TMC.

Installing SMS

You can perform the following tasks:

• Validate the VMware Environment, page 3

• Obtain the vSMS Software from the TMC, page 3

• Obtain the vSMS Certification String, page 4

• Deploy the vSMS Software, page 4

• Start the vSMS Software, page 5

• Configure the SMS Server, page 5

• Install the Client Software, page 6

• Complete Initial Setup, page 6

Validate the VMware Environment

Before you deploy the vSMS, ensure your VMware environment is based on vSphere and meets the system requirements described in ”VMware vSphere Environment” and ”Minimum System Requirements” on page 1.

NOTE: You cannot adjust physical resource settings during initial deployment of the vSMS. To adjust the

settings, first deploy the vSMS, and then use the vSphere Client to modify the physical resource settings. Note that once disk size is increased it cannot be decreased.

Obtain the vSMS Software from the TMC

The vSMS is distributed to customers through the TMC. Download the software from the TMC and store it in a location accessible to the VMware management application vCenter. Perform the following steps to obtain the software:

1. In a Web browser, open https://tmc.tippingpoint.com

2. Select Releases, and then select Software > SMS > Virtual SMS (vSMS).

3. On the vSMS Software Packages page, select the appropriate vSMS software entry.

4. Note the MD5 checksum displayed in the “Message” area of the Software Details page. You will

compare it against the checksum you generate after you download the file to your local system.

5. Click Download.

6. Accept the End User License Agreement, and save the file to a storage location that is accessible to the VMware vCenter where you want to deploy the vSMS.

7. Generate an MD5 checksum against your local copy of the .zip file, and then compare it against the MD5 checksum shown on the TMC.

, and log in to the TMC.

vSMS Getting Started 3

8. Unzip the downloaded vSMS software package. The software package expands into two files, both of which are needed to deploy the vSMS. Their

names are similar in format to the following:

vsms-4.0.vSMS.xxxx.ovf

vsms-disk1-4.0.vSMS.xxxx.vmdk

Obtain the vSMS Certification String

1. After HP TippingPoint receives your product purchase order, you receive a physical registration card that is mailed to you soon after the order is placed.

2. Use the information on the card to contact HP TippingPoint by email to obtain your unique vSMS certification string.

3. HP TippingPoint emails your unique vSMS certification string to you. When you receive the certification string, you can begin deployment of the vSMS.

Deploy the vSMS Software

The vSMS is a virtual appliance compressed and packaged according to the VMware Open Virtualization Format (OVF). The vSMS contains a ready-to-configure instance of SMS. When the vSMS is deployed the SMS software running in the virtual appliance operates in the same manner as if it were running on a physical SMS appliance.

CAUTION: Deployment of the vSMS .ovf file must be performed through VMware vCenter Server.

Deploying it directly through ESX/ESXi utilities is not supported.

1. Use the VMware vSphere Client to log on to the VMware vCenter Server that manages the ESX/ESXi host where you want to deploy the vSMS.

2. Select the host where you want to deploy the vSMS. When you deploy the vSMS be sure to deploy it onto an ESX/ESXi host that has network access to the

devices you want the vSMS appliance to manage.

3. Use the following steps to deploy the vSMS .ovf file:

a. Click File > Deploy OVF Template. b. Locate the *.ovf file you obtained when you unzipped the vSMS software package, and then

click Next. c. Verify the template details, and click Next. d. Specify a name and a location for the vSMS, and then click Next. e. Specify a host/cluster where you want to deploy the vSMS, and then click Next. f. Select a datastore for the vSMS, and click Next.

NOTE: If the storage page of the OVF deployment wizard indicates the host where you are

installing the vSMS appliance does not provide sufficient disk space, you should deploy the vSMS appliance to a different host that does have sufficient disk capacity. If you do not have another host where you can deploy the vSMS appliance, select thin provisioning format in the next step.

4 Installation

g. Choose the format for storing the virtual disks: thin or thick provisioning format. h. Select a Destination Network to which to map the source network in the OVF template.

i. Enter the SMS certification string; cut and paste the certification string from the email you

received from HP TippingPoint into the field in the deployment wizard, and then click Next.

NOTE: In some cases the certification string you paste into this field will not be displayed. When

you click Next, be sure to verify the string that appears on the summary screen.

CAUTION: The vSMS requires a valid certification string during the startup procedure; the

certification string must match the string from HP TippingPoint. If you open the string in an application before you copy and paste it into the OVF deployment wizard, make sure the application does not insert carriage returns, new line characters, or other unseen characters.

j. Verify the deployment settings on the summary screen, and then click Finish.

4. After the OVF deployment process completes, right-click the vSMS virtual machine and select Edit Settings.

5. Confirm that the first network interface is assigned to the virtual network with access to the security devices you want the vSMS to manage, and then click OK.

Start the vSMS Software

While logged in to the VMware vCenter Server, perform the following steps to launch the vSMS and open a console.

1. Expand the datacenter and datastore folders until you see the virtual machine where you installed vSMS.

2. Right-click the vSMS and select Power > Power On.

3. As the virtual machine starts, monitor the vCenter Recent Tasks pane to ensure it completes the power-on process.

4. When the virtual machine is powered on, you can open a console to monitor the booting of the guest operating system. To do this, right-click the virtual machine and select Open Console.

Configure the SMS Server

After powering up the vSMS, the SMS Out-of-Box (OBE) Setup Wizard prompts you to perform basic tasks to configure the system. Perform the following steps:

1. Log on to the SMS server as

SuperUser

(no password).

2. Read and accept the end-user license agreement to continue.

3. If needed, select a language for a different keyboard layout.

4. Specify a security level (0 – 2) and create a new Super User administrator account and password.

5. Specify the network type, SMS management IP address, network mask, and optional default gateway.

6. Specify a host name to describe the SMS. If desired, enter the optional host location and system contact information.

7. Modify the timekeeping option by enabling NTP Client for your time zone.

8. Modify server options for SSH, HTTPS, HTTP, and SNMP.

9. As an optional step, you can configure a Network Management System to monitor and receive SNMP traps.

10. Configure email contact information.

vSMS Getting Started 5

Install the Client Software

The SMS Client can be installed on a virtual machine or on a physical machine. Do not install the SMS Client on the virtual machine where the vSMS exists.

Supported operating systems include Windows XP, Windows Vista, Windows 7, Linux and Mac OSX.

1. Start your Web browser.

2. In your browser Address bar, enter the IP address or host name of your SMS Appliance. For example:

https://123.45.67.89

3. Log in with the Super User account that you created during the SMS Server setup.

4. On the SMS Welcome page, select the client that is compatible with your computer software or click the Client Installation link in the navigation pane.

5. Complete the client download and installation instructions provided on the SMS Client Installation Web page.

6. Double-click the SMS Client icon on your desktop to start the SMS Client.

7. Specify the IP address or fully qualified host name of your SMS Server.

8. Enter the Super User account user name and password that you created during the SMS server setup.

9. Click Login.

At the bottom of the dialog box, the status message Attempting to connect is displayed. In a few seconds, the message Connected, logging in is displayed. After a successful login, the SMS Dashboard is displayed.

Complete Initial Setup

Use the following tasks to complete the initial setup and begin using the SMS to manage your HP TippingPoint devices.

Add a Device

1. On the SMS toolbar, click Devices.

2. In the Devices navigation pane, select All Devices.

3. On the Devices screen, click New Device.

4. In the New Device wizard, specify the following device information:

• IP Address

• Username for a SuperUser account defined on a device

• Password associated with the SuperUser account

•Device Group

•Device Type

5. Click OK. At the bottom of the dialog, a status bar displays blinking green icons and status messages. After each device is added, the dialog box closes automatically.

6. To add multiple devices, repeat the previous steps.

7. In th e Devices window, check the health of the devices by verifying that the Health status indicator is green.

Download a Digital Vaccine

1. In a Web browser, open https://tmc.tippingpoint.com

, and log in to the TMC. See ”Before You

Begin” on page 3.

2. Select Releases, and then select Digital Vaccine > Digital Vaccines.

3. Locate the file you want to download, generally the most recent version, and click the corresponding Download link for that file.

After the file downloads, it is displayed on the DV Inventory tab in the Profiles workspace.

You can now activate this digital vaccine, distribute it to managed devices, view details, or delete the DV package. For more information, see the HP TippingPoint Security Management System User Guide.

6 Installation

Where to Go Next

The SMS acts a central console where you can manage multiple HP TippingPoint devices, products, and services. After the initial setup, you can begin monitoring and managing your HP TippingPoint systems.

Make sure all HP TippingPoint devices that you add to the SMS are configured or enabled to accept SMS management. Refer to device product documentation for information about preparing a device for SMS management.

For IPS devices, the SMS performs most of the tasks that are also available from the IPS Local Security Manager (LSM) application. When an IPS device is enabled for SMS control, the device is exclusively controlled by the SMS. You can unmanage devices in the SMS.

For complete information about managing HP TippingPoint systems, see the HP TippingPoint SMS User Interface Guide, or the SMS online Help.

NOTE: To access the SMS command line interface (CLI) you must log in with the Super User account. The

SuperUser account used to access the CLI must have the following authorization: SMS_ACCESS_CLI. For more information about using the CLI, see the HP TippingPoint Security Management System Command Line Interface Reference.

vSMS Getting Started 7

8 Installation