McAfee M-1250, Network Security Platform Platform Manual

Special Topics Guide—In-line Sensor Deployment

revision 1.0

McAfee® Network Security Platform

Network Security Sensor
version 6.0

McAfee®

Network Protection

Industry-leading network security solutions

COPYRIGHT

Copyright ® 2001 - 2009 McAfee, Inc. All Rights Reserved. No part of this publication may be reproduced, transmitted, transcribed, stored in a retrieval system, or translated into
any language in any form or by any means without the written permission of McAfee, Inc., or its suppliers or affiliate companies.

TRADEMARKS

ACTIVE FIREWALL, ACTIVE SECURITY, ACTIVESECURITY (AND IN KATAKANA), ACTIVESHIELD, CLEAN-UP, DESIGN (STYLIZED E), DESIGN (STYLIZED N),
ENTERCEPT, EPOLICY ORCHESTRATOR, FIRST AID, FOUNDSTONE, GROUPSHIELD, GROUPSHIELD (AND IN KATAKANA), INTRUSHIELD, INTRUSION PREVENTION
THROUGH INNOVATION, McAfee, McAfee (AND IN KATAKANA), McAfee AND DESIGN, McAfee.COM, McAfee VIRUSSCAN, NET TOOLS, NET TOOLS (AND IN KATAKANA),
NETSCAN, NETSHIELD, NUTS & BOLTS, OIL CHANGE, PRIMESUPPORT, SPAMKILLER, THREATSCAN, TOTAL VIRUS DEFENSE, VIREX, VIRUS FORUM, VIRUSCAN,
VIRUSSCAN, VIRUSSCAN (AND IN KATAKANA), WEBSCAN, WEBSHIELD, WEBSHIELD (AND IN KATAKANA) are registered trademarks or trademarks of McAfee, Inc. and/or
its affiliates in the US and/or other countries. The color red in connection with security is distinctive of McAfee brand products. All other registered and unregistered trademarks
herein are the sole property of their respective owners.

LICENSE AND PATENT INFORMATION
License Agreement

NOTICE TO ALL USERS: CAREFULLY READ THE APPROPRIATE LEGAL AGREEMENT CORRESPONDING TO THE LICENSE YOU PURCHASED, WHICH SETS FORTH
THE GENERAL TERMS AND CONDITIONS FOR THE USE OF THE LICENSED SOFTWARE. IF YOU DO NOT KNOW WHICH TYPE OF LICENSE YOU HAVE ACQUIRED,
PLEASE CONSULT THE SALES AND OTHER RELATED LICENSE GRANT OR PURCHASE ORDER DOCUMENTS THAT ACCOMPANIES YOUR SOFTWARE PACKAGING
OR THAT YOU HAVE RECEIVED SEPARATELY AS PART OF THE PURCHASE (AS A BOOKLET, A FILE ON THE PRODUCT CD, OR A FILE AVAILABLE ON THE WEB SITE
FROM WHICH YOU DOWNLOADED THE SOFTWARE PACKAGE). IF YOU DO NOT AGREE TO ALL OF THE TERMS SET FORTH IN THE AGREEMENT, DO NOT INSTALL
THE SOFTWARE. IF APPLICABLE, YOU MAY RETURN THE PRODUCT TO McAfee OR THE PLACE OF PURCHASE FOR A FULL REFUND.

License Attributions

This product includes or may include:
* Software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/). * Cryptographic software written by Eric A. Young and software written by

Tim J. Hudson. * Some software programs that are licensed (or sublicensed) to the user under the GNU General Public License (GPL) or other similar Free Software licenses
which, among other rights, permit the user to copy, modify and redistribute certain programs, or portions thereof, and have access to the source code. The GPL requires that for
any software covered under the GPL, which is distributed to someone in an executable binary format, that the source code also be made available to those users. For any such
software covered under the GPL, the source code is made available on this CD. If any Free Software licenses require that McAfee provide rights to use, copy or modify a software
program that are broader than the rights granted in this agreement, then such rights shall take precedence over the rights and restrictions herein. * Software originally written by
Henry Spencer, Copyright 1992, 1993, 1994, 1997 Henry Spencer. * Software originally written by Robert Nordier, Copyright (C) 1996-7 Robert Nordier. * Software written by
Douglas W. Sauder. * Software developed by the Apache Software Foundation (http://www.apache.org/). A copy of the license agree ment for this software can be found at
www.apache.org/licenses/LICENSE-2.0.txt. * International Components for Unicode ("ICU") Copyright (C) 1995-2002 International Business Machines Corporation and others. *
Software developed by CrystalClear Software, Inc., Copyright (C) 2000 CrystalClear Software, Inc. * FEAD(R) Optimizer(R) technology, Copyright Netopsystems AG, Berlin,
Germany. * Outside In(R) Viewer Technology (C) 1992-2001 Stellent Chicago, Inc. and/or Outside In(R) HTML Export, (C) 2001 Stellent Chicago, Inc. * Software copyrighted by
Thai Open Source Software Center Ltd. and Clark Cooper, (C) 1998, 1999, 2000. * Software copyrighted by Expat maintainers. * Software copyrighted by The Regents of the
University of California, (C) 1996, 1989, 1998-2000. * Software copyrighted by Gunnar Ritter. * Software copyrighted by Sun Microsystems, Inc., 4150 Network Circle, Santa Clara,
California 95054, U.S.A., (C) 2003. * Software copyrighted by Gisle Aas. (C) 1995-2003. * Software copyrighted by Michael A. Chase, (C) 1999-2000. * Software copyrighted by
Neil Winton, (C) 1995-1996. * Software copyrighted by RSA Data Security, Inc., (C) 1990-1992. * Software copyrighted by Sean M. Burke, (C) 1999, 2000. * Software copyrighted
by Martijn Koster, (C) 1995. * Software copyrighted by Brad Appleton, (C) 1996-1999. * Software copyrighted by Michael G. Schwern, (C) 2001. * Software copyrighted by Graham
Barr, (C) 1998. * Software copyrighted by Larry Wall and Clark Cooper, (C) 1998-2000. * Software copyrighted by Frodo Looijaard, (C) 1997. * Software copyrighted by the Python
Software Foundation, Copyright (C) 2001, 2002, 2003. A copy of the license agreement for this software can be found at www.python .org. * Software copyrighted by Beman
Dawes, (C) 1994-1999, 2002. * Software written by Andrew Lumsdaine, Lie-Quan Lee, Jeremy G. Siek (C) 1997-2000 University of Notre Dame. * Software copyrighted by Simone
Bordet & Marco Cravero, (C) 2002. * Software copyrighted by Stephen Purcell, (C) 2001. * Software developed by the Indiana University Extreme! Lab
(http://www.extreme.indiana.edu/). * Software copyrighted by International Business Machines Corporation and others, (C) 1995-2003. * Software developed by the University of
California, Berkeley and its contributors. * Software developed by Ralf S. Engelschall <[e-mail]> for use in the mod_ssl project (http:// www.modssl.org/). * Software
copyrighted by Kevlin Henney, (C) 2000-2002. * Software copyrighted by Peter Dimov and Multi Media Ltd. (C) 2001, 2002. * Software copyrighted by David Abrahams, (C) 2001,

2002. See http://www.boost.org/libs/bind/bind.html for documentation. * Software copyrighted by Steve Cleary, Beman Dawes, Howard Hinnant & John Maddock, (C) 2000. *
Software copyrighted by Boost.org, (C) 1999-2002. * Software copyrighted by Nicolai M. Josuttis, (C) 1999. * Software copyrighted by Jeremy Siek, (C) 1999-2001. * Software
copyrighted by Daryle Walker, (C) 2001. * Soft ware copyrighted by Chuck Allison and Jeremy Siek, (C) 2001, 2002. * Software copyrighted by Samuel Krempp, (C) 2001. See
http://www.boost.org for updates, documentation, and revision history. * Software copyrighted by Doug Gregor ([e-mail]), (C) 2001, 2002. * Software copyrighted by
Cadenza New Zealand Ltd., (C) 2000. * Software copyrighted by Jens Maurer, (C) 2000, 2001. * Software copyrighted by Jaakko Järvi ([e-mail]), (C) 1999, 2000. *
Software copyrighted by Ronald Garcia, (C) 2002. * Software copyrighted by David Abrahams, Jeremy Siek, and Daryle Walker, (C) 1999-2001. * Softw are copyrighted by Stephen
Cleary ([e-mail]), (C) 2000. * Software copyrighted by Housemarque Oy <http://www.housemarque.com>, (C) 2001. * Software copyrighted by Paul Moore, (C)

1999. * Software copyrighted by Dr. John Maddock, (C) 1998-2002. * Software copyrighted by Greg Colvin and Beman Dawes, (C) 1998, 1999. * Software copyrighted by Peter
Dimov, (C) 2001, 2002. * Software copyrighted by Jeremy Siek and John R. Bandela, (C) 2001. * Software copyrighted by Joerg Walter and Mathias Koch, (C) 2000-2002. *
Software copyrighted by Carnegie Mellon University (C) 1989, 1991, 1992. * Software copyrighted by Cambridge Broadband Ltd., (C) 2001-2003. * Software copyrighted by
Sparta, Inc., (C) 2003-2004. * Software copyrighted by Cisco, Inc and Information Network Center of Beijing University of Posts and Telecommunications, (C) 2004. * Software
copyrighted by Simon Josefsson, (C) 2003. * Software copyrighted by Thomas Jacob, (C) 2003-2004. * Software copyrighted by Advanced Software Engineering Limited, (C)

2004. * Software copyrighted by Todd C. Miller, (C) 1998. * Software copyrighted by The Regents of the University of California, (C) 1990, 1993, with code derived from software
contributed to Berkeley by Chris Torek.

Issued DECEMBER 2009 / Special Topics Guide—In-line Sensor Deployment

700-2381-00/ 1.0 - English

Contents

Preface ........................................................................................................... v

Introducing McAfee Network Security Platform............................................................................. v

About this Guide............................................................................................................................ v

Conventions used in this guide ..................................................................................................... v

Related Documentation.................................................................................................................vi

Contacting Technical Support......................................................................................................vii

Chapter 1 What is inline mode?................................................................... 1

Benefits of running inline............................................................................................................... 1

Chapter 2 Inline deployment walkthrough ................................................. 3

Chapter 3 Determine your high availability strategy................................. 4

Failover, or High-Availability.......................................................................................................... 4

Fail-open or fail-closed functionality.............................................................................................. 5

Chapter 4 Install and cable the Sensor....................................................... 6

Cable the Fast Ethernet monitoring ports...................................................................................... 7

Cable the Gigabit Ethernet monitoring ports................................................................................. 7

Cable a failover pair ...................................................................................................................... 7

Configure the Sensor monitoring ports.......................................................................................... 8

About Sensor port configuration.............................................................................................8

Chapter 5 Failover: configure two Sensors in inline mode .................... 11

Create a Failover Pair ................................................................................................................. 11

Download configuration, signature set, and software updates to the Sensor......................12

Chapter 6 Configure policies..................................................................... 13

Tune your policies.......................................................................................................................13

About false positives and "noise"................................................................................................14

Incorrect identification..........................................................................................................14

Correct identification; significance subject to usage policy..................................................14

Correct identification; significance subject to user sensitivity (also known as noise)...........14

Chapter 7 Block attacks ............................................................................. 16

Methods for blocking attacks....................................................................................................... 16

Block exploit traffic ...................................................................................................................... 16

How blocking works for exploit traffic...................................................................................17

Verify dropped exploit attacks using the Threat Analyzer....................................................17

Block DoS traffic.......................................................................................................................... 17

How blocking works for DoS traffic ......................................................................................18

Verify blocked DoS attacks using the Threat Analyzer........................................................18

Drop DoS Attacks from the Threat Analyzer........................................................................18

Block using ACLs........................................................................................................................18

Utilize traffic normalization .......................................................................................................... 19

Blocking based on configured TCP & IP Settings....................................................................... 20

Blocking of IP-spoofed packets................................................................................................... 20

Chapter 8 Troubleshooting........................................................................ 21

iii

Verify that traffic is flowing through the Sensor........................................................................... 21

Verify failover pair creation success............................................................................................ 21

show.....................................................................................................................................21

status....................................................................................................................................21

show failover-status .............................................................................................................22

downloadstatus ....................................................................................................................22

Index............................................................................................................. 23

iv

Preface

This preface provides a brief introduction to the product, discusses the information in this
document, and explains how this document is organized. It also provides information such
as, the supporting documents for this guide and how to contact McAfee Technical Support.

Introducing McAfee Network Security Platform

McAfee® Network Security Platform [formerly McAfee® IntruShield®] delivers the most
comprehensive, accurate, and scalable Network Access Control (NAC), network Intrusion
Prevention System (IPS) and Network Threat Behavior Analysis (NTBA) for mission-critical
enterprise, carrier, and service provider networks, while providing unmatched protection
against spyware and known, zero-day, and encrypted attacks.

McAfee Network Threat Behavior Analysis Appliance provides the capability of monitoring
network traffic by analyzing NetFlow information flowing through the network in real time,
thus complementing the NAC and IPS capabilities in a scenario in which McAfee Network
Security Sensor, NAC Sensor, and NTBA Appliance are installed and managed through a
single Manager.

About this Guide

This guide describes the process of deploying Network Security Sensors (Sensors) in inline
mode

. The information in this guide details best practices for inline mode configuration,

information on attack blocking, and inline troubleshooting options.
This guide assumes that the reader has a working understanding of McAfee Network

Security Platform products, including McAfee Network Security Manager (nsm) and
McAfee Network Security Sensors (Sensors).

Conventions used in this guide

This document uses the following typographical conventions:

Convention Example

Terms that identify fields, buttons,
tabs, options, selections, and
commands on the User Interface
(UI) are shown in
font.

Menu or action group selections
are indicated using a right angle
bracket.

Arial Narrow bold

Service field on the Properties tab specifies the

The
name of the requested service.

Select My Company > Admin Domain > Summary.

v

McAfee® Network Security Platform 6.0

Convention Example

Preface

Procedures are presented as a
series of numbered steps.

Names of keys on the keyboard
are denoted using UPPER CASE.

Text such as syntax, key words,
and values that you must type
exactly are denoted using

Courier New

font.

Variable information that you must
type based on your specific
situation or environment is shown

italics.

in
Parameters that you must supply

are shown enclosed in angle
brackets.

Information that you must read
before beginning a procedure or
that alerts you to negative
consequences of certain actions,
such as loss of data is denoted
using this notation.

Information that you must read to
prevent injury, accidents from
contact with electricity, or other
serious consequences is denoted
using this notation.

1. On the Configuration tab, click Backup.

Press ENTER.

setup and then press ENTER.

Type:

Type: S

ensor-IP-address and then press

ENTER.

set Sensor ip <A.B.C.D>

Caution:

Warning:

Notes that provide related, but
non-critical, information are
denoted using this notation.

Related Documentation

The following documents and on-line help are companions to this guide. Refer to Quick
Tour for more information on these guides

• Quick Tour

• Installation Guide

• Upgrade Guide

• Getting Started Guide

• IPS Deployment Guide

• Manager Configuration Basics Guide

• I-1200 Sensor Product Guide

• I-1400 Sensor Product Guide

• I-2700 Sensor Product Guide

• I-3000 Sensor Product Guide

Note:

vi

McAfee® Network Security Platform 6.0

• I-4000 Sensor Product Guide

• I-4010 Sensor Product Guide

• M-1250/M-1450 Sensor Product Guide

• M-1250/M-1450 Quick Start Guide

• M-2750 Sensor Product Guide

• M-2750 Quick Start Guide

• M-3050/M-4050 Sensor Product Guide

• M-3050/M-4050 Quick Start Guide

• M-6050 Sensor Product Guide

• M-6050 Quick Start Guide

• M-8000 Sensor Product Guide

• M-8000 Quick Start Guide

• Gigabit Optical Fail-Open Bypass Kit Guide

• Gigabit Copper Fail-Open Bypass Kit Guide

• 10 Gigabit Fail-Open Bypass Kit Guide

• M-8000/M-6050/M-4050/M-3050 Slide Rail Assembly Procedure

• M-2750 Slide Rail Assembly Procedure

• M-series DC Power Supply Installation Procedure

• Administrative Domain Configuration Guide

• Manager Server Configuration Guide

• CLI Guide

• Device Configuration Guide

• IPS Configuration Guide

• NAC Configuration Guide

• Integration Guide

• System Status Monitoring Guide

• Reports Guide

• Custom Attack Definitions Guide

• Central Manager Administrator's Guide

• Best Practices Guide

• Troubleshooting Guide

• Special Topics Guide—Sensor High Availability

• Special Topics Guide—Virtualization

• Special Topics Guide—Denial-of-Service

• NTBA Appliance Administrator's Guide

• NTBA Monitoring Guide

• NTBA Appliance T-200 Quick Start Guide

• NTBA Appliance T-500 Quick Start Guide

Preface

Contacting Technical Support

If you have any questions, contact McAfee for assistance:

vii

McAfee® Network Security Platform 6.0

Online

Contact McAfee Technical Support http://mysupport.mcafee.com.
Registered customers can obtain up-to-date documentation, technical bulletins, and quick

tips on McAfee's 24x7 comprehensive KnowledgeBase. In addition, customers can also
resolve technical issues with the online case submit, software downloads, and signature
updates.

Phone

Technical Support is available 7:00 A.M. to 5:00 P.M. PST Monday-Friday. Extended 24x7
Technical Support is available for customers with Gold or Platinum service contracts.
Global phone contact numbers can be found at McAfee
http://www.mcafee.com/us/about/contact/index.html page.

Note: McAfee requires that you provide your GRANT ID and the serial number of

your system when opening a ticket with Technical Support. You will be provided with
a user name and password for the online case submission.

Preface

Contact Information

viii

C HAPTER 1

What is inline mode?

Inline monitoring mode provides prevention of attacks by enabling Security Administrators
to select the types of attacks/traffic to drop, thus preventing the negative end-system
impact common with today's network attacks. Inline mode is achieved when Network
Security Sensor is placed directly in the path of a network segment, becoming,
essentially, a “bump in the wire,” with packets flowing through Sensor. In this mode, the
Sensor inspects all traffic at wire-speed and can prevent network attacks by dropping
malicious traffic in real time—the Sensor actually ends the attacking transmission before it
can reach and impact the target. Preventative actions can operate at a highly granular
level, including the automated dropping of DoS traffic intended for a specific host.

When operating in inline mode, network segments are connected to two wire-matched
Sensor ports (For example: peer ports 1A and 1B), and packets are examined in real time
as they pass through the Sensor. In this mode, a packet comes in through the first
interface of the pair of the Sensor and out the second interface of the pair. The packet is
sent to the second interface of the pair unless that packet is being denied or modified by a
signature.

As of release 2.1.7, Sensor ports are configured by default for monitoring in inline mode;
that is, connected inline on a network segment (For example: between a switch and a
router or two switches). A Sensor with 2.1.7 or later software will initially come online with
its peer ports configured in pairs and in inline mode.

Note: This change will not override user-configured settings. Deployed Sensors

upgraded to 2.1.7 or later and will retain their user-configured settings.

Benefits of running inline

The benefits to using Sensors in inline mode are:

Protection/Prevention. Prevention is a feature unique to inline mode. When running inline,

•

a Sensor can drop malicious packets and not pass them through the network. This

acts sort of like an “adaptive firewall,” with your detection policy dictating what is

dropped. Furthermore, when dropping packets, Network Security Platform is very

precise and granular. The Sensor can drop only those packets it identifies as

malicious or all of the packets related to that flow (a choice that is user configurable).

Packet “scrubbing.” In addition to dropping malicious traffic, Network Security Platform

•

can

attacker may be using to try to evade detection. Current IDS products are susceptible

to these techniques, and an example of this attempt is IP fragment and TCP segment

overlaps. The Sensor can reassemble the IP fragments and TCP segments and

enforce a reassembly mode of the user’s choice to accept either the old or the new

data.

Processing at wire-speed. Sensors are able to process packets at wire rates.

•

scrub—or normalize—traffic to take out any ambiguities in protocols that the

1

McAfee® Network Security Platform 6.0

In inline mode, the Sensor logically acts as a transparent repeater with minimal

latency for packet processing. Unlike bridges, routers, or switches, the Sensor does

not need to learn MAC addresses or keep an ARP cache or a routing table.

What is inline mode?

2