HP HP-UX IPSec Administrator's Guide

Download

HP-UX IPSec version A.02.00

Administrator’s Guide

HP-UX 11i version 1 and HP-UX 11i version 2

Manufacturing Part Number : J4256-90009

June 2004

United States

Legal Notices

The information in this document is subject to change without notice.

Warranty

The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

U.S. Government License

Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.

Copyright rights reserved. Reproduction, adaptation, or translation of this document without prior written permission is prohibited, except as allowed under the copyright laws.

Trademark Notice

UNIX countries, licensed exclusively through The Open Group.



1999-2004 Hewlett-Packard Development Company L.P. All



is a registered trademark in the United States and other

Preface: About This Document

1. HP-UX IPSec Overview

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Authentication Header (AH) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Transport and Tunnel Modes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Transport Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Tunnel Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Encapsulating Security Payload (ESP). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

ESP Encryption. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Transport and Tunnel Modes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Transport Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Tunnel Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

ESP with Authentication and Encryption. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Authenticated ESP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Nested ESP in AH. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

IPv6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Internet Key Exchange (IKE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Security Associations (SAs) and IKE Phases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Generating Shared Keys: Diffie-Hellman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

IKE Primary Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

IKE Preshared Key Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Digital Signatures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Re-using Negotiations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

IKE Automatic Re-keying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Manual Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

HP-UX IPSec Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Host-to-Host Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Host-to-Gateway Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Host-to-Host Tunnel Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Gateway-to-Gateway Topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

HP-UX IPSec Configuration and Management Features . . . . . . . . . . . . . . . . . . . . . . . 46

Contents

2. Installing HP-UX IPSec

HP-UX IPSec Product Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Disk Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Security Certificate Configuration Utility Requirements . . . . . . . . . . . . . . . . . . . . . 51

Step 1: Verifying HP-UX IPSec Installation and Configuration Prerequisites . . . . . . 52

iii

Contents

Step 2: Loading the HP-UX IPSec Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Step 3: Setting the HP-UX IPSec Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Re-establishing the HP-UX IPSec Password. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Step 4: Completing Post-Installation Migration Requirements . . . . . . . . . . . . . . . . . . 56

3. Configuring HP-UX IPSec

Maximizing Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Bypass List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Strong End System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Using ipsec_config . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

General Syntax Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Argument Delimiters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Line Continuation Character (\) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

ipsec_config add . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

ipsec_config batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

Batch File Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

Batch File Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

ipsec_config delete . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

ipsec_config show . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Profile File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Using a Profile File with a Batch File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

Profile File Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

Creating a Customized Profile File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

Dynamic Configuration Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Dynamic Deletions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

nocommit Argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Configuration Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Step 1: Configuring Host IPSec Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Policy Order and Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

default Host IPSec Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Automatic Priority Increment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

ipsec_config add host Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

host_policy_name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

-source and -destination ip_addr[/prefix[/port_number|service_name]] . . . . . . . . 71

-protocol protocol_id . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

-priority priority_number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

Contents

-tunnel tunnel_policy_name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

-action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

-flags flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Host IPSec Policy Configuration Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Step 2: Configuring Tunnel IPSec Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

ipsec_config add tunnel Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

tunnel_policy_name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

-tsource and -tdestination tunnel_address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

-source and -destination ip_addr[/prefix[/port_number|service_name]] . . . . . . . . 83

-protocol protocol_id . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

-action transform_list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

lifetime_seconds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

lifetime_kbytes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Tunnel IPSec Policy Configuration Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Step 3: Configuring IKE Policies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Policy Order and Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Automatic Priority Increment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

ipsec_config add ike Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

ike_policy_name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

-remote ip_addr[/prefix] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

-priority priority_number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

-authentication authentication_type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

-group 1|2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

-hash MD5|SHA1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

-life lifetime_seconds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

-maxqm max_quick_modes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

ipsec_config add IKE Command Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Step 4: Configuring Preshared Keys Using Authentication Records . . . . . . . . . . . . . . 95

Remote Multi-homed Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Configuring IKE ID Information with Preshared Keys . . . . . . . . . . . . . . . . . . . . . . . 95

ipsec_config add auth Syntax. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

auth_name. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

ip_addr[/prefix] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

preshared_key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Authentication Record Configuration Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Step 5: Configuring Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Step 6: Configuring the Bypass List (Local IPv4 Addresses) . . . . . . . . . . . . . . . . . . . 101

Logical Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Contents

Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

Maximizing Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

ipsec_config add bypass Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

ip_address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Bypass Configuration Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Step 7: Verify Batch File Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

Step 8: Committing the Batch File Configuration and Verifying Operation . . . . . . . 105

Step 9: Configuring HP-UX IPSec to Start Automatically . . . . . . . . . . . . . . . . . . . . . 109

ipsec_config add startup Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Step 10: Creating Backup Copies of the Batch File and Configuration Database . . . 111

Baltimore Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

VeriSign Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

4. Using Certificates with HP-UX IPSec

Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Security Certificates and Public Key Cryptography. . . . . . . . . . . . . . . . . . . . . . . . . 115

Public Key Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Security Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Digital Signatures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

IKE Public Key Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Using VeriSign Certificates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

VeriSign Certificate Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Step 1: Verifying Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

Step 2: Configuring Web Proxy Server Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 121

Step 3: Registering the Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Step 4: Requesting and Receiving Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Using Baltimore Certificates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Baltimore Certificate Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Step 1: Verifying Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

Step 2: Requesting the Baltimore Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Step 3: Configuring the Baltimore Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Configuring Authentication Records with IKE IDs . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Configuring Authentication Records with Certificate-Based Authentication. . . . . 135

Determining the IPv4 Address in the SubjectAlternativeName . . . . . . . . . . . . . 136

Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

-ltype local_id_type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

-lvalue local_id. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

-rtype remote_id_type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

-rid remote_id . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

Retrieving the Certificate Revocation List (CRL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

VeriSign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

Baltimore. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

Manually Retrieving a CRL for VeriSign or Baltimore . . . . . . . . . . . . . . . . . . . . . . 143

5. Troubleshooting HP-UX IPSec

IPSec Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

Establishing Security Associations (SAs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

Internal Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

Outbound Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

Inbound Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

Establishing Tunnel Security Associations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

Processing Inbound Tunnel Packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

Troubleshooting Utilities Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

Getting General Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Getting SA Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Getting Policy Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Getting Interface Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

Viewing and Configuring Audit Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

Enabling and Disabling Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

Troubleshooting Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

Checking Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

Isolating HP-UX IPSec Problems from Upper-layer Problems . . . . . . . . . . . . . . . . 162

Checking Policy Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Using ipsec_policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Examining the Policy Cache and Policy Entries . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Configuring HP-UX IPSec Auditing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Audit Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Audit Files and Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Dynamically Setting Audit Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Configuring Startup Audit Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Viewing Audit Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Contents

vii

Contents

Filtering Audit File Output by Entity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

Reporting Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

Troubleshooting Scenarios. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

HP-UX IPSec Incorrectly Passes Packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

HP-UX IPSec Incorrectly Attempts to Encrypt/Authenticate Packets . . . . . . . . . . 172

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

HP-UX IPSec Attempts to Encrypt/Authenticate and Fails. . . . . . . . . . . . . . . . . . . 172

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

ISAKMP/MM SA Negotiation Fails (Main Mode processing failed, MM negotiation

timeout) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

ISAKMP Primary Authentication with Preshared Key Fails . . . . . . . . . . . . . . . . . 176

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

ISAKMP Primary Authentication Fails with Certificates . . . . . . . . . . . . . . . . . . . . 176

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

ISAKMP/MM SA Negotiation Succeeded, IPSec/QM SA Negotiation Fails (Quick Mode

processing failed, QM negotiation timeout). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

viii

Contents

Manual Keys Fail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

Invalid SADB_ADD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

STREAMS Logging Messages and Additional Audit File Entries . . . . . . . . . . . . 180

HP-UX Will Not Start (ipsec_admin -start Fails) . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

Corrupt or Missing Configuration Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Using ipsec_migrate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Using the Skeleton Database File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Autoboot is Not Working Properly. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Administrator Cannot Get a Local VeriSign Certificate . . . . . . . . . . . . . . . . . . . . . 185

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Security Policy Database Limit Exceeded (Kernel Policy Cache Threshold reached or

Kernel Policy Cache Threshold exceeded) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

Symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

6. HP-UX IPSec and IPFilter

IPFilter and IPSec Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

IPSec UDP Negotiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

When Traffic Appears to be Blocked. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

Allowing Protocol 50 and Protocol 51 Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

IPSec Gateways . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

Contents

7. HP-UX IPSec and HP-UX Mobile IPv6

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Mobile Node and Home Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Care-of Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Correspondent Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Home Agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Home Agents and Basic Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

Route Optimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

Securing Mobile IPv6 with HP-UX IPSec . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

Binding Messages Between the Home Agent and Mobile Node. . . . . . . . . . . . . . 204

Return Routability Messages Between the Home Agent and Mobile Node . . . . . 205

Prefix Discovery Packets Between the Home Agent and Mobile Node . . . . . . . . 206

Payload Packets Routed Through the Home Agent. . . . . . . . . . . . . . . . . . . . . . . . 206

Configuration Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

Understanding Gateway IPSec Policies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

Using Manual Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

Selecting Encryption Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

Using the HP-UX Strong Random Number Generator. . . . . . . . . . . . . . . . . . . . . 209

Troubleshooting Manual Key Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

Configuration Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

Step 1: (Required) Securing Binding Messages Between the Home Agent and Mobile

Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

host_policy_name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

-source home_agent_addr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

-destination mn_home_addr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

-protocol MH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

-priority priority_number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

-action transform_name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

-in and -out manual_key_sa_specification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

Step 2: (Recommended) Securing Return Routability Messages Routed Through the

Home Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

Step 2A: Return Routability Messages: Configuring the Gateway IPSec Policy for

Home Agent - Correspondent Node Segments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

Step 2B: Return Routability Messages: Configuring the Gateway IPSec Policy for

Home Agent - Mobile Node Segments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

Contents

Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

Step 2C: Return Routability Messages: Configuring the Home Agent - Mobile Node

Tunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

Step 3: (Recommended) Securing Prefix Discovery Messages Between the Home Agent

and Mobile Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

host_policy_name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

-source home_agent_addr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

-destination mn_home_addr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

-priority priority_number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

-action transform_name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

-flags MIPV6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

-in and -out manual_key_sa_specification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

Step 4: (Optional) Securing Payload Packets Routed Through the Home Agent . . . . 222

Step 4A: Payload Packets: Configuring the Gateway IPSec Policy for Home Agent -

Correspondent Node Segments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

Step 4B: Payload Packets: Configuring the Gateway IPSec Policy for Home Agent -

Mobile Node Segments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

Step 4C: Payload Packets: Configuring the Home Agent - Mobile Node Tunnel . . 225

Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

Mobile IPv6 Configuration Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Binding Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Return Routability Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Return Routability Gateway IPSec Policies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

Return Routability Tunnel IPSec Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

(Optional) Prefix Discovery Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

(Optional) Payload Messages Routed Through the Home Agent. . . . . . . . . . . . . . . 229

Payload Gateway IPSec Policies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

Payload Tunnel IPSec Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

Batch File Template. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

8. HP-UX IPSec and MC/ServiceGuard

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

Using HP-UX IPSec with MC/ServiceGuard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

Package Clients Not Using HP-UX IPSec A.01.07 or Later . . . . . . . . . . . . . . . . . 238

Configuration Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

Contents

Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

MC/ServiceGuard Heartbeat Requirement and Recommendation . . . . . . . . . . . . . 240

Configuration Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

Step 1: Configuring a Common HP-UX IPSec Password . . . . . . . . . . . . . . . . . . . . . . . 243

Step 2: Configuring HP-UX Host IPSec Policies for MC/ServiceGuard . . . . . . . . . . . 244

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

Determining MC/ServiceGuard Cluster Information . . . . . . . . . . . . . . . . . . . . . . . . 245

Configuring Host IPSec Policies for Package Addresses . . . . . . . . . . . . . . . . . . . . . 245

Configuring PASS Host IPSec Policies for Heartbeat IP Addresses . . . . . . . . . . . . 245

Private Dedicated Heartbeat Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

Configuring Host IPSec Policies for MC/ServiceGuard Quorum Server . . . . . . . . . 247

Cluster Node IPSec Policies for Quorum Server . . . . . . . . . . . . . . . . . . . . . . . . . . 247

Quorum Server IPSec Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

Configuring Host IPSec Policies for Remote Command Execution . . . . . . . . . . . . . 248

Cluster Node IPSec Policies for Remote Command Execution. . . . . . . . . . . . . . . 248

Remote Command Client Host IPSec Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

Configuring Host IPSec Policies for ServiceGuard Manager . . . . . . . . . . . . . . . . . . 251

Cluster Node Host IPSec Policies for ServiceGuard Manager . . . . . . . . . . . . . . . 252

ServiceGuard Manager Host IPSec Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Configuring Host IPSec Policies for Cluster Object Manager (COM) . . . . . . . . . . . 253

Cluster Node Host IPSec Policies for COM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

COM System Host IPSec Policies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

Summary: MC/ServiceGuard Port Numbers and Protocols . . . . . . . . . . . . . . . . . . . 254

Step 3: Configuring HP-UX IPSec IKE policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

Cluster IKE policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

Cluster Client IKE policies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

Step 4: Configuring Authentication Records for Preshared Keys . . . . . . . . . . . . . . . . 258

Preshared Key Configuration on Cluster Nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

Preshared Key Configuration on Client Nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

Preshared Keys Configuration on Cluster Nodes . . . . . . . . . . . . . . . . . . . . . . . . . 259

Preshared Keys Configuration on Client1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

Preshared Keys Configuration on Client2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

Step 5: Configuring Authentication Records for Certificates . . . . . . . . . . . . . . . . . . . 261

Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

Authentication Records and IKE ID Information. . . . . . . . . . . . . . . . . . . . . . . . . . . 261

xii

Cluster Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262

Cluster Clients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262

Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

IKE ID Configuration on Cluster Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

IKE ID Configuration on Client1 and Client2. . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

Step 6: Verifying and Testing the HP-UX IPSec Configuration . . . . . . . . . . . . . . . . . 265

Step 7: Configuring HP-UX IPSec Start-up Options . . . . . . . . . . . . . . . . . . . . . . . . . . 266

Step 8: Distributing HP-UX IPSec Configuration Files. . . . . . . . . . . . . . . . . . . . . . . . 267

Baltimore Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

VeriSign Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

Step 9: Configuring MC/ServiceGuard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

Cluster Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

Package Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

Package Control Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

Monitor Script Polling Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270

Step 10: Starting HP-UX IPSec and MC/ServiceGuard. . . . . . . . . . . . . . . . . . . . . . . . 271

Adding a Node to a Running Cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271

9. HP-UX IPSec and Linux

Limitations of HP-UX IPSec Interoperating with Linux FreeSwan . . . . . . . . . . . . . . 275

Configuration Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

Contents

A. Product Specifications

IPSec RFCs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

RFC 3776 Mandatory Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

Product Restrictions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

ISAKMP Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

IPv4 ICMP Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

IPv6 ICMP Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

HP-UX IPSec Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284

Comparative Key Lengths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284

Authentication Algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284

Encryption Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

ESP-DES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

ESP-DES-HMAC-MD5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

ESP-DES-HMAC-SHA1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

ESP-3DES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

ESP-3DES-HMAC-MD5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

xiii

Contents

ESP-3DES-HMAC-SHA1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

ESP-AES128 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

ESP-AES128-HMAC-MD5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

ESP-AES128-HMAC-SHA1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

ESP-NULL-HMAC-MD5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

ESP-NULL-HMAC-SHA1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

Transform Lifetime Negotiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

B. Migrating from Previous Versions of HP-UX IPSec

Pre-Installation Migration Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

MD5 Version Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

Migrating from Versions Prior to A.01.03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

Not Re-using Configuration Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

Post-Installation Migration Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

Configuration File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

C. HP-UX IPSec Configuration Examples

Example 1: telnet Between Two Systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

Apple Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296

Host IPSec Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296

IKE Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

Authentication Record with Preshared Key . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

Banana Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

Example 2: Authenticated ESP with Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

Carrot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

Host IPSec Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

IKE Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

Authentication Record . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

Example 3: Host to Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Blue Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Host IPSec Policy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Tunnel IPSec Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

IKE Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

Authentication Record . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

Example 4: Manual Keys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

Dog Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

xiv

Contents

Cat Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

Contents

xvi

Tab les

Table 1. Publishing History Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii

Table 3-1. ipsec_config Service Names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .72

Table 3-2. ipsec_config Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76

Table 3-3. ipsec_config add host Flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .78

Table 5-1. Getting General Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .156

Table 5-2. Getting SA Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .156

Table 5-3. Getting Policy Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .156

Table 5-4. Getting Interface Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .158

Table 5-5. Viewing and Configuring Audit Information . . . . . . . . . . . . . . . . . . . . . .158

Table 5-6. Enabling and Disabling Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159

Table 8-1. MC/ServiceGuard Port Numbers and Protocols . . . . . . . . . . . . . . . . . . . .254

Table A-1. Supported IPSec RFCs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .279

Table A-2. AH and ESP Algorithms and Key Lengths. . . . . . . . . . . . . . . . . . . . . . . .284

xvii

Tables

xviii

Figures

Figure 1-1. Symmetric Key Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30

Figure 1-2. AH in Transport Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

Figure 1-3. AH in Tunnel Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Figure 1-4. Symmetric Key Cryptosystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33

Figure 1-5. ESP Encryption in Transport Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35

Figure 1-6. ESP in Tunnel Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35

Figure 1-7. Authenticated ESP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36

Figure 1-8. Nested ESP in AH. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37

Figure 1-9. SA Establishment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

Figure 1-10. Diffie-Hellman Key Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40

Figure 1-11. IPSec Host-to-Host Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

Figure 1-12. Host-to-Gateway (VPN) Topology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

Figure 1-13. Host-to-Host Tunnel Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

Figure 1-14. IPSec Gateway-to-Gateway Topology. . . . . . . . . . . . . . . . . . . . . . . . . . . .45

Figure 3-1. Bypass List Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .102

Figure 4-1. VeriSign PKI Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .119

Figure 4-2. VeriSign SubjectAlternativeName. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .136

Figure 5-1. Security Associations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .147

Figure 5-2. Outbound Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .149

Figure 6-1. IPFilter and IPSec. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .191

Figure 6-2. IPFilter Scenario One . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .191

Figure 6-3. IPFilter Scenario Two . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .193

Figure 6-4. Scenario Three . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .195

Figure 6-5. Packet with Encrypted TCP Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .196

Figure 6-6. Packet with IPSec-Encrypted TCP Data . . . . . . . . . . . . . . . . . . . . . . . . .196

Figure 6-7. Scenario Four . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .197

Figure 7-1. Mobile IPv6 Basic Operation: Correspondent Node to Mobile Node . . .202 Figure 7-2. Mobile IPv6 Basic Operation: Mobile Node to Correspondent Node . . .203

Figure 7-3. Mobile IPv6 Route Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .203

Figure 7-4. Gateway IPSec Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .208

Figure 7-5. Mobile IPv6 Home Test Init and Home Test Packets . . . . . . . . . . . . . . .214

Figure 8-1. MC/ServiceGuard Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .237

Figure C-1. Example 1: telnet AB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .296

xix

Figures

Figure C-2. Example 1: telnet BA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .296

Figure C-3. Example 2: Network IPSec Policy with Exceptions . . . . . . . . . . . . . . . .299

Figure C-4. Host to Gateway Configuration Example . . . . . . . . . . . . . . . . . . . . . . . .302

Preface: About This Document

This document describes how to install, configure, and troubleshoot HP-UX IPSec.

The document printing date and part number indicate the document’s current edition. The printing date will change when a new edition is printed. Minor changes may be made at reprint without changing the printing date. The document part number will change when extensive changes are made.

Document updates may be issued between editions to correct errors or document product changes. To ensure that you receive the updated or new editions, you should subscribe to the appropriate product support service. See your HP sales representative for details.

The latest version of this document can be found online at http://docs.hp.com.

Intended Audience

This document is intended for system and network administrators responsible for installing, configuring, and managing HP-UX IPSec. Administrators are expected to have knowledge of HP-UX and networking concepts, commands, and configuration.

This document is not a tutorial.

New and Changed Documentation in This Edition

The documentation reflects the following changes to the HP-UX IPSec product:

• IPSec policies, bypass list and startup parameters are now configured using the

ipsec_config command-line utility. The ipsec_config utility also supports batch files.

The ipsec_mgr GUI is still used to configure security certificates.

• The IPSec policy, preshared key and startup configuration information is now stored

in a configuration database, /var/adm/ipsec/config.db. The policy configuration file (default /var/adm/ipsec/policies.txt), preshared key file (/var/adm/ipsec/pskeys.txt), and startup file (/etc/rc.config.d/ipsecconf) files are no longer used.

• HP-UX IPSec was enhanced to support dynamic configuration updates. Administrators can update the configuration without stopping and re-starting HP-UX IPSec.

• HP-UX IPSec now supports manual keys for IPSec Security Associations.

• HP-UX IPSec was enhanced to secure Mobile IPv6 packets with manual keys when the local system is a Mobile IPv6 Home Agent.

• HP-UX IPSec can act as a gateway (IP router) and forward IP packets, but only for HP-UX Mobile IPv6.

• In previous releases, there was only one type of IPSec policy, which contained both host and tunnel IPSec information. There are now separate host IPSec policies and tunnel IPSec policies. There are also gateway IPSec policies, which are supported only for HP-UX Mobile IPv6.

• ISAKMP policies are now referred to as IKE policies.

xxi

• The default Oakley group (Diffie-Hellman group) is now 2.

• Preshared keys are configured in authentication records.

• Administrators can now configure preshared keys for remote subnets.

• IKE ID parameters can now be configured for IKE negotiations when using preshared keys.

• Certificate IDs are configured as IKE ID information in authentication records. The authentication records are indexed and searched by remote IP address. There is no longer a certificate ID record for the local system (127.0.0.0).

•The ipsec_report utility supports the following new options:

• -entity (used with the -audit option): The -entity option allows you to specify

one or more entities when displaying an audit file (-audit). This allows you to selectively display audit records logged by specify entities.

• -host: The -host option displays IPSec policies loaded by the policy daemon.

• -ike: The -ike option displays IKE policies loaded by the policy daemon.

• -gateway: The -gateway option displays gateway IPSec policies loaded by the

policy daemon.

• -tunnel: The -tunnel option displays tunnel IPSec policies loaded by the policy

daemon. The ipsec_report options -ipsec and -isakmp are still supported, but only for

backwards compatibility and are not documented. The ipsec_report option

-ipsec reports host IPSec policies (it is now equivalent to the -host option). The ipsec_report option -isakmp reports IKE policies (it is now equivalent to the

-ike option).

•The ipsec_policy utility now allows you to specify a direction for the packet parameters.

•The ipsec_admin utility supports the following new options to set general operating parameters:

• -spd_soft: The -spd_soft option allows you to specify the “soft” limit for the

size of the Security Policy Database (SPD). The SPD is the HP-UX IPSec runtime policy database, with cached policy decisions for packet descriptors (five-tuples consisting of exact, non-wildcard source IP address, destination IP address, protocol, source port, and destination port).

• -spd_hard: The -spd_hard option allows you to specify the “hard” limit for the

size of the SPD.

• -spi_min: The -spi_min option allows you to specify the lower bound for

inbound, dynamic key Security Parameters Index (SPI) numbers.

• -spi_max: The -spi_max option allows you to specify the upper bound for

inbound, dynamic key Security Parameters Index (SPI) numbers.

• IPv6 IKE functionality, formerly provided by the daemon ikmpdv6, is now provided by ikmpd. The ikmpdv6 daemon is no longer shipped with the product.

xxii

Publishing History

Table 1 Publishing History Details

Document

Manufacturing

Part Numb er

J4256-90009 11i version 1 (B.11.11)

J4256-90005 11i version 1 (B.11.11) A.01.07 August 2003

J4256-90003 11i version 2 (B.11.23) A.01.06 July 2003

J4256-90001 11.0

J4255-9011 11.0

Operating Systems

Supported

11i version 2 (B.11.23)

11.04 11i version 1 (B.11.11)

Supported

Product

Version s

A.02.00 June 2004

A.01.05 August 2002

A.01.04 December

Publication

Date

2001

What’s in This Document

HP-UX IPSec Administrator’s Guide is divided into several chapters, and each contains information about installing, configuring, or troubleshooting HP-UX IPSec. The appendices also contain supplemental information.

Chapter 1 HP-UX IPSec Overview Use this chapter to learn about product

features and topologies.

Chapter 2 Installing HP-UX IPSec Use this chapter to verify installation

prerequisites and to learn how to install the product.

Chapter 3 Configuring HP-UX IPSec Use this chapter to learn how to

configure HP-UX IPSec.

Chapter 4 Using Certificates with HP-UX IPSec Use this chapter to learn

how to configure HP-UX IPSec to use security certificates.

Chapter 5 Troubleshooting HP-UX IPSec Use this chapter to learn how to

troubleshoot HP-UX IPSec, what to do for common problems, how to report problems, and how to use the IPSec troubleshooting tools.

Chapter 6 HP-UX IPSec and IPFilter Use this chapter to learn how to use

HP-UX IPSec with IPFilter.

Chapter 7 HP-UX IPSec and HP-UX Mobile IPv6 Use this chapter to learn

how to use HP-UX IPSec with HP-UX Mobile IPv6.

Chapter 8 HP-UX IPSec and MC/ServiceGuard Use this chapter to learn how

to use HP-UX IPSec with MC/ServiceGuard.

Chapter 9 HP-UX IPSec and Linux Use this chapter to learn how to use HP-UX

IPSec with Linux FreeSwan.

Appendix A Product Specifications Use this appendix to learn the product

specifications: supported RFCs, product limitations and restrictions.

xxiii

Appendix B Migrating from Previous Versions of HP-UX IPSec Use this

appendix to find out how to migrate from previous versions of HP-UX IPSec.

Appendix C HP-UX IPSec Configuration Examples Use this appendix to see

configuration parameters for simple topologies.

Typographical Conventions

This document uses the following conventions.

audit (5) An HP-UX manpage. In this example, audit is the name and 5 is the

section in the HP-UX Reference. On the web and on the Instant Information CD, it may be a hot link to the manpage itself. From the HP-UX command line, you can enter “man audit” or “man 5 audit” to view the manpage. See man (1).

Book Title The title of a book. On the web and on the Instant Information CD, it

may be a hot link to the book itself.

KeyCap The name of a keyboard key. Note that Return and Enter both refer to

the same key.

Emphasis Text that is emphasized.

Bold Text that is strongly emphasized.

Bold The defined use of an important word or phrase.

ComputerOut Text displayed by the computer. UserInput Commands and other text that you type. Command A command name or qualified command phrase.

variable

[] The contents are optional in formats and command descriptions. If the

{} The contents are required in formats and command descriptions. If the

... The preceding element may be repeated an arbitrary number of times.

| Separates items in a list of choices.

The name of a variable that you may replace in a command or function

or information in a display that represents several possible values.

contents are a list separated by |, you must choose one of the items.

1 HP-UX IPSec Overview

Chapter 1 25

HP-UX IPSec Overview

This chapter describes HP-UX IPSec features and topologies. It contains the following sections:

• “Introduction” on page 27

• “Authentication Header (AH)” on page 29

• “Encapsulating Security Payload (ESP)” on page 33

• “Internet Key Exchange (IKE)” on page 38

• “HP-UX IPSec Topologies” on page 43

Chapter 126

HP-UX IPSec Overview

Introduction

The IP security (IPSec) protocol suite was defined by the Internet Engineering Task Force (IETF) to provide security for IP networks. The IPSec protocol suite provides the following security services for IP networks:

• Data Integrity

Guarantee data consistency; prevent unauthorized creation, modification, or deletion of data between source and destination.

• Authentication

Ensure that the data received is the same as the data sent and that the claimed sender is the actual sender.

• Confidentiality

Provide data privacy such that only the intended recipients of the data know what is being sent. The sender encrypts (encodes) the data using an encryption algorithm and key (bit string). The output is ciphertext that is difficult to decode without knowing the key.

• Application-transparent Security

IPSec security headers are inserted between the standard IP protocol header and the upper-layer data (such as a TCP packet). Any network service that uses IP (such as telnet, FTP or sendmail) or user application that uses IP (TCP BSD Socket or XTI Streams application) can use IPSec without modification.

IPSec traffic can also pass transparently through existing IP routers.

IPSec functionality can be divided into the following categories:

• Authentication Header (AH) for data integrity and authentication.

• Encapsulating Security Payload (ESP) header for data confidentiality, data integrity, and data authentication. The ESP header also includes a sequence number that provides a form of replay protection.

Chapter 1 27

HP-UX IPSec Overview

Introduction

• Internet Key Exchange (IKE) protocol, for generating and distributing cryptography keys for ESP and AH. IKE also authenticates the identity of the remote system, so AH and authenticated ESP with IKE keys provides data origin authentication.

• Manual Keys, an alternative to IKE. Instead of dynamically generating and distributing cryptography keys for ESP and AH, the cryptography keys are static and manually distributed. Manual keys are typically used only when the remote system does not support IKE, or in Mobile IPv6 topologies.

Chapter 128

HP-UX IPSec Overview

Authentication Header (AH)

The IPSec Authentication Header (AH) provides integrity and authentication but no privacy--the IP data is not encrypted. The AH contains an authentication value based on a symmetric-key hash function.

Symmetric key hash functions are a type of cryptographic hash function that take the data and a key as input to generate an authentication value. Cryptographic hash functions are usually one-way functions, so that starting with a hash output value, it is difficult to create an input value that would generate the same output value. This makes it difficult for a third party to intercept a message and replace it with a new message that would generate the same authentication value.

Symmetric key hash functions are also known as shared key hash functions because the sender and receiver must use the same (symmetric) key for the hash functions. In addition, the key must only be known by the sender and receiver, so this class of hash functions is sometimes referred to as secret key hash functions.

In the example below, the sender uses the plaintext and shared secret key to calculate an authentication value and sends the authentication value with the plaintext. The recipient computes its own authentication value using the same shared secret key and the plaintext. The recipient then compares the result with the transmitted authentication value. If

Chapter 1 29

HP-UX IPSec Overview

Authenticati on Header (AH)

the values match, the recipient is assured that the sender knows the same secret key, confirming the identity of the sender. The recipient is also assured that the data was not altered during transit.

Figure 1-1 Symmetric Key Authentication

Host A

Host B

Plaintext

authentication value

Shared Key

hash

Plaintext

authentication value

HP-UX IPSec supports the following algorithms for AH:

• HMAC-SHA1 (Hashed Message Authentication Code-Secure Hash Algorithm 1, 128-bit key)

• HMAC-MD5 (HMAC-Message Digest 5, 160-bit key)

authentication value

Plaintext

hash

Shared Key

(compare)

Transport and Tunnel Modes

The IPSec headers (AH and ESP) can be used in transport mode or tunnel mode.

Transport Mode

In transport mode, IPSec inserts the AH header after the IP header. The IP data and header are used to calculate the AH authentication value. Mutable fields in the IP header (fields that need to change in transit), such as “hop count,” and “time to live,” are assigned a zero value before IPSec calculates the authentication value, so the actual value of the mutable fields are not authenticated.

Chapter 130

HP-UX IPSec Overview

Authentication Header (AH)

IPv6 In IPv6 AH transport mode, IPSec inserts the AH after the following headers and extensions:

• the basic IPv6 header

• hop-by-hop options

• any destination options needed to interpret the AH header

• routing extensions

• fragment extensions

The items listed below follow the AH:

• any destination options needed only for the “final” destination and not needed to interpret the AH header

• the IP data or payload (e.g., TCP or UDP packet)

The entire packet is used to calculate the authentication value. Mutable and unpredictable fields and options, such as timestamp and traceroute options, are assigned a zero value before calculating the authentication value.

Figure 1-2 AH in Transport Mode

Chapter 1 31

HP-UX IPSec Overview

Authenticati on Header (AH)

Tunnel Mode

IPv6 In IPv6 AH tunnel mode, the packet layout is the same as IPv4 AH tunnel mode, except that the original and new (outer) IP headers may include header extensions.

Figure 1-3 AH in Tunnel Mode

Chapter 132

Encapsulating Security Payload (ESP)

The IPSec Encapsulating Security Payload (ESP) provides data privacy. The ESP protocol also defines an authenticated format that provides

data authentication and integrity, with data privacy (described in “Authenticated ESP” on page 36).

ESP Encryption

ESP takes the data carried by IP, such as a TCP packet, and encrypts it using an encryption algorithm and cryptographic key. The output is ciphertext that is difficult to decode without knowing the key. The receiving IPSec ESP entity uses an associated decryption algorithm and the same key to extract the original data.

Figure 1-4 Symmetric Key Cryptosystem

Shared Cryptographic Key

HP-UX IPSec Overview

Encapsulating Secur ity Payload (E SP)

Encryption

Plaintext

The cryptography used by ESP is referred to as symmetric key cryptography or shared key cryptography because the sender and receiver must use the same key. In addition, the key must only be known by the sender and receiver, so this class of cryptography is sometimes referred to as secret key cryptography.

HP-UX IPSec supports the following encryption algorithms for ESP:

• DES-CBC (Data Encryption Standard Cipher Block Chaining Mode, 56-bit key length)

• 3DES-CBC (Triple-DES CBC, three encryption iterations, each with a different 56-bit key)

• AES128-CBC (Advanced Encryption Standard CBC, 128-bit key length)

Chapter 1 33

Host A

Ciphertext

Decryption

Host B

Plaintext

HP-UX IPSec Overview

Encapsulating Secur ity Payload (E SP)

AES128-CBC is the most secure form of encryption for HP-UX IPSec. AES128-CBC encryption throughput rates are comparable to or better than DES-CBC and 3DES-CBC. For more information about HP-UX IPSec performance, refer to the HP-UX IPSec Sizing and Performance document available at www.docs.hp.com.

DES-CBC has been cracked (data encoded by DES has been decoded by a third party).

For added security, use ESP with authentication, as described in “ESP with Authentication and Encryption” on page 36.

Transport and Tunnel Modes

The ESP header can be used in transport mode or tunnel mode.

Transport Mode

In transport mode, the original IP header is followed by the ESP header. Only the upper-layer (e.g., TCP, UDP, IGMP) is encrypted. The IP header is not encrypted.

IPv6 In IPv6 ESP transport mode, IPSec inserts the ESP header after the following headers and extensions:

• the basic IPv6 header

• hop-by-hop options

• any destination options needed to interpret the ESP header

• routing extensions

• fragment extensions

The items listed below follow the ESP header and are encrypted:

• any destination options needed only for the “final” destination and not needed to interpret the ESP header

Chapter 134

• the IP data or payload (e.g., TCP or UDP packet)

Figure 1-5 ESP Encryption in Transport Mode

Tunnel Mode

In tunnel mode, IPSec encloses, or encapsulates, the original IP datagram, including the original IP header, within a second IP datagram. All of the original IP datagram, including the original header, is encrypted. If ESP is used in tunnel mode on gateways, the outer, unencrypted IP header will contain the IP addresses of the gateways, and the inner, encrypted IP header will contain the ultimate IP source and destination addresses. This prevents eavesdroppers from detecting or analyzing traffic between the ultimate source and destination addresses.

HP-UX IPSec Overview

Encapsulating Secur ity Payload (E SP)

IPv6 In IPv6 ESP tunnel mode, the packet layout is the same as IPv4 ESP tunnel mode, except that the original and new (outer) IP headers may include header extensions.

Figure 1-6 ESP in Tunnel Mode

Chapter 1 35

HP-UX IPSec Overview

Encapsulating Secur ity Payload (E SP)

ESP with Authentication and Encryption

The ESP encryption algorithms by themselves not provide authentication or guarantee data integrity, so you should use ESP encryption with an authentication and data integrity service. There are two ways to do this:

• use the authenticated ESP format

• nest ESP within AH (nested ESP in AH)

Authenticated ESP

With authenticated ESP, IPSec encrypts the payload using one symmetric key, then calculates an authentication value for the encrypted data using a second symmetric key and the HMAC-SHA1 or HMAC-MD5 algorithm. The ESP authentication value is appended to the end of the packet. The recipient computes its own authentication value for the encrypted data using the second symmetric key and the same algorithm. The recipient compares the result with the transmitted authentication value. If the values match, the recipient then decrypts the encrypted portion of the packet with the first symmetric key and extracts the original data.

Figure 1-7 Authenticated ESP

Chapter 136

Nested ESP in AH

An ESP packet can be nested within an AH packet. For example, a 3DES-CBC ESP packet can be nested within an HMAC-MD5 packet. IPSec uses 3DES-CBC to build an ESP packet with the payload data encrypted using a symmetric key. IPSec then nests the ESP packet within an AH packet, using a second symmetric key. All the contents of the packet are authenticated, except the mutable fields of the IP header.

IPv6

The packet layouts and procedures for authenticated ESP and nested ESP in AH are the same for IPv6, except that the IP headers may include header extensions.

Figure 1-8 Nested ESP in AH

HP-UX IPSec Overview

Encapsulating Secur ity Payload (E SP)

Chapter 1 37

HP-UX IPSec Overview

Internet Key Exchange (IKE)

Before IPSec sends authenticated or encrypted IP data, both the sender and receiver must agree on the protocols, encryption algorithms and keys to use. HP-UX IPSec uses the Internet Key Exchange (IKE) protocol to negotiate the encryption and authentication methods, and generate shared encryption keys. The IKE protocol also provides primary authentication - verifying the identity of the remote system before negotiating the encryption algorithm and keys.

The IKE protocol is a hybrid of three other protocols: Internet Security Association and Key Management Protocol (ISAKMP), Oakley, and Versatile Secure Key Exchange Mechanism for Internet protocol (SKEME). ISAKMP provides a framework for authentication and key exchange, but does not define them (neither authentication nor key exchange). The Oakley protocol describes a series of modes for key exchange and the SKEME protocol defines key exchange techniques.

Security Associations (SAs) and IKE Phases

A Security Association (SA) is a secure communication channel and its parameters, such as the encryption algorithm, keys and lifetime. There are two SA negotiation phases within ISAKMP, which are sometimes referred to by the Oakley modes used to establish the SAs. The general flow of the IKE protocol is as follows:

1. ISAKMP Phase One (Main Mode, MM)

• Negotiate and establish an ISAKMP SA, a secure communication

channel for further IKE communication.

The two systems generate a Diffie-Hellman shared value (described below) that is used as the base for a symmetric (shared) key, and further IKE communication is encrypted using this symmetric key.

• Verify the remote system’s identity (primary authentication)

2. ISAKMP Phase Two (Quick Mode, QM)

Chapter 138

Using the secure communication channel provided by the ISAKMP/MM SA, negotiate one or more SAs for IPSec transforms (AH or ESP). A Phase Two negotiation typically negotiates two SAs for an IPSec transform: one for inbound and one for outbound traffic.

Figure 1-9 SA Establishment

HP-UX IPSec Overview

Internet Key Exchange (IKE)

IPSec

ISAKMP

NodeA

Inbound

Outbound

IPSec/QM SAs

ISAKMP phase 1

ISAKMP phase 2

IP traffic

( secured by IPSec AH/ESP )

Outbound

Inbound

IPSec/QM SAs

IPSec

ISAKMP

NodeB

Generating Shared Keys: Diffie-Hellman

SAs use a symmetric key to encrypt communication. This symmetric key is based on a shared value generated using the Diffie-Hellman algorithm.

With Diffie-Hellman key generation, each party generates two numbers, one public and one private. These values are based on a selected, well-known numeric base, or “Diffie-Hellman group.” The two parties exchange public values (this exchange may occur via an insecure channel). Each party then uses its private value and the other party’s

Chapter 1 39

HP-UX IPSec Overview

Internet Key Exchange (IKE)

public value to generate a new value. Because of the mathematical properties of the numbers, each party will generate the same value, which can then be used as a symmetric key.

Figure 1-10 Diffie-Hellman Key Generation

Node A

Node B

Step 1

Step 2

Step 3

Public Val ue

Private

Value

Shared Secret Value

A & B select Diffie-Hellman Group

Public Value

Private

Value

Private Value A combined with Public Value B

Private Value B combined with Public Value A

Shared

Diffie-Hellman is vulnerable to attacks where a third-party intercepts messages between the sender and receiver and assumes the identity of the other party. Because of this, Diffie-Hellman is used with some form of authentication to ensure that symmetric keys are established between correct parties.

In summary, if two entities use the same, well-known Diffie-Hellman group, they can publicly exchange values and generate the same shared value that they can use as a symmetric key, or use as a base for a symmetric key. Diffie-Hellman should be used with some form of authentication.

Secret Value

IKE Primary Authentication

IKE must authenticate the identities of the systems using the Diffie-Hellman algorithm. This process is known as primary authentication. HP-UX IPSec IKE can use two primary authentication methods:

•Preshared keys

Chapter 140

HP-UX IPSec Overview

Internet Key Exchange (IKE)

• Digital Signatures

IKE Preshared Key Authentication

With preshared key authentication, you must manually configure the same, shared symmetric key on both systems, a preshared key. The preshared key is used only for the primary authentication. The two negotiating entities then generate dynamic shared keys for the IKE SAs and IPSec/QM SAs.

Preshared keys do not require a Certificate Authority or Public Key Infrastructure.

Digital Signatures

Digital signatures are based on security certificates, and are managed using a Public Key Infrastructure (PKI). HP-UX IPSec supports the following security certificates from the following products:

• VeriSign Managed PKI (formerly VeriSign OnSite for VPNs)

• Baltimore UniCERT 3.5

For more information on using certificate-based authentication for IKE, see Chapter 4, “Using Certificates with HP-UX IPSec,” on page 113.

Re-using Negotiations

For efficiency, you can specify that a single ISAKMP Phase One (ISAKMP/MM) SA can be used to negotiate multiple ISAKMP Phase Two (IPSec/QM) negotiations.

Conversely, you can specify that each Phase One SA can be used for only one ISAKMP Phase Two negotiation. The IKE daemon will create a new ISAKMP SA for each IPSec SA negotiation. This can provide a feature known as Perfect Forward Secrecy (PFS) with key and identity protection. With PFS, the compromise (exposure) of one key exposes only the data protected by that key.

IKE Automatic Re-keying

The IKE protocol also allows HP-UX IPSec to dynamically negotiate new IPSec keys rather than exposing the same key for long periods. You can configure key lifetimes based on time or number of bytes sent.

Chapter 1 41

HP-UX IPSec Overview

Manual Keys

Manual keys are an alternative to IKE. Instead of using IKE to dynamically generate and distribute cryptography keys for ESP and AH, the cryptography keys are static and manually distributed. Because manual keys are static, using them is less secure than using IKE. Manual keys are typically used only when the remote system does not

support IKE, or in Mobile IPv6 topologies.

Chapter 142

HP-UX IPSec Topologies

IPSec can be employed between hosts (that is, end nodes), between gateways, or between a host and a gateway in an IP network. HP-UX IPSec can only be installed on end nodes.

Installing HP-UX IPSec on an HP-UX system that is a router or gateway is not supported, except when the system is used as an HP-UX Mobile IPv6 Home Agent. See Chapter 7, “HP-UX IPSec and HP-UX Mobile IPv6,” on page 199 for more information.

Host-to-Host Topology

Two end hosts can run HP-UX IPSec locally to protect communication between them, with or without intermediate gateways.

Figure 1-11 IPSec Host-to-Host Topology

HP-UX IPSec Overview

HP-UX IPSec Topologies

HostA HP-UX IPSec

Internet

Secure channel

HostB HP-UX IPSec

Host-to-Gateway Topology

In situations where the local subnet is a trusted network, you can use HP-UX IPSec between an end host and a gateway and create a secure Virtual Private Network (VPN).

Figure 1-12 Host-to-Gateway (VPN) Topology

Chapter 1 43

HP-UX IPSec Overview

HP-UX IPSec Topologies

NOTE In an Host-to-Gateway topology, the gateway cannot be an HP-UX

system unless the gateway is an HP-UX Mobile IPv6 Home Agent, and the gateway functionality is used only to forward packets for Mobile IPv6.

Host-to-Host Tunnel Topology

Two end hosts with HP-UX IPSec protection can configure a tunnel policy that securely protects traffic between them. The tunnel adds extra protection; an intruder cannot see the real IP headers of packets traveling between the hosts.

The host-to-host tunnel topology is commonly used in an iSCSI environment.

Figure 1-13 Host-to-Host Tunnel Topology

Chapter 144

Gateway-to-Gateway Topology

Two hosts each reside upon insecure networks (such as insecure intranets). These hosts need to communicate securely over an insecure public network (such as the Internet). HP-UX IPSec can be used over a tunnel between two (non-HP) IPSec gateways to provide additional end-to-end security.

Figure 1-14 IPSec Gateway-to-Gateway Topology

Tunnel

HP-UX IPSec Overview

HP-UX IPSec Topologies

HostA HP-UX IPSec

Non-HP

Gateway

IPSec

Secure channel

Internet

Non-HP

Gateway

IPSec

HostX HP-UX IPSec

Chapter 1 45

HP-UX IPSec Overview

HP-UX IPSec Configurati on and Management Featur es

HP-UX IPSec Configuration and Management Features

The HP-UX IPSec product includes the configuration and management features listed below.

• Easy-to-use configuration utilities You configure HP-UX IPSec using ipsec_config, which allows

batch mode operation. To configure security certificates, use ipsec_mgr, which has a graphical user interface (GUI) and online help.

• Packet-based configuration

You control IPSec behavior by defining packet filters in IPSec policies. An IPSec policy contains a packet filter definition and list of actions or transforms (pass, discard, use ESP or AH) to apply to the packets. The packet filter definition contains the following fields:

— local IP address

— local address prefix length (for subnet addresses)

— remote IP address

— remote address prefix length (for subnet addresses)

— upper-layer protocol (such as TCP, UDP. or ICMP)

— local TCP or UDP port number

— remote TCP or UDP port number

You can also select a network service for the filter, such as telnet, instead of the upper-layer protocol and port numbers.

• Bypass IPv4 address configuration

You can configure HP-UX IPSec to bypass, or ignore, local IPv4 interfaces that you do not need to secure. This feature is useful for internal networks where most traffic passes in clear text and only specific applications need to be secured.

• Configuration test utility

Chapter 146

HP-UX IPSec Overview

HP-UX IPSec Configuration and Management Features

The ipsec_policy utility takes a packet definition (local and remote IP addresses, upper-layer protocol, local and remote port numbers) as input and reports the IPSec policy that HP-UX IPSec would apply to packets matching the definition.

• Audit logging

HP-UX IPSec maintains an audit log of events, including events that may indicate attempts to compromise network security.

• Data reporting utility The ipsec_report utility reports IPSec runtime data, including

information about SAs and entries in the audit log.

• Status reporting utility The ipsec_admin utility reports the status of HP-UX IPSec

components. The ipsec_admin utility also performs general administrative functions, such as starting and stopping HP-UX IPSec, setting the audit level, and deleting or resetting runtime data.

Chapter 1 47

HP-UX IPSec Overview

HP-UX IPSec Configurati on and Management Featur es

Chapter 148

2 Installing HP-UX IPSec

Chapter 2 49

Installing HP-UX IPSec

This chapter describes installation prerequisites and procedures for installing HP-UX IPSec software. It contains the following sections:

• “HP-UX IPSec Product Requirements” on page 51

• “Step 1: Verifying HP-UX IPSec Installation and Configuration Prerequisites” on page 52

• “Step 2: Loading the HP-UX IPSec Software” on page 53

• “Step 3: Setting the HP-UX IPSec Password” on page 55

• “Step 4: Completing Post-Installation Migration Requirements” on page 56

Chapter 250

Installing HP-UX IPSec

HP-UX IPSec Product Requirements

Prior to installing the HP-UX IPSec product, check that your system can accommodate the following product requirements.

Disk Requirements

The total size of the disk space required for the HP-UX IPSec product is 112 Mbytes. Requirements for variable-length user files are listed below:

• Configuration database file (/var/adm/ipsec/config.db): minimum of 50 kbytes per policy file. Most configuration database files will be approximately 1 Mbyte or less. An extremely large configuration database (thousands of entries) may be as large as 5 Mbytes.

• Audit file: This file can grow very fast if Informative auditing is enabled. HP recommends 1 Mbyte for the Alerts and Errors level of logging, 5 Mbytes for the Warnings level, and 200 or more Mbytes for the Informative message level. Informative auditing could generate 3–5 Mbytes per hour. Audit files should be kept in a separate directory or file system. The default directory is: /var/adm/ipsec/.

Security Certificate Configuration Utility Requirements

If you are using security certificates for IKE authentication, you must use the ipsec_mgr configuration utility. The ipsec_mgr utility requires a graphical display device. This can be a graphics monitor, an X terminal display, a PC with X Server software installed, or a Linux workstation running an X Server.

Chapter 2 51

Installing HP-UX IPSec

Step 1: Veri f y ing HP-UX IPSec Install a t ion and Configuration Prerequisit es

Step 1: Verifying HP-UX IPSec Installation and Configuration Prerequisites

1. Verify that the operating system version is HP-UX 11i version 1 (B.11.11) or HP-UX 11i version 2 (B.11.23).

To obtain information about the OS, execute the command:

uname -a

2. Check the latest HP-UX IPSec release note for patch information.

To obtain information about a patch, execute the command:

swlist -i

3. Be sure you have root access and are designated the network security administrator.

4. If you are migrating from a previous version of HP-UX IPSec, be sure you have completed the procedures in Appendix B, “Pre-Installation Migration Instructions” on page 289.

5. If you are using security certificates (RSA signatures) for IKE authentication, be sure you have access to a graphical display, either on your system, a remote HP-UX workstation, or a remote display device.

6. If you are using security certificates for IKE authentication, ensure that you are registered and can communicate with a certificate authority.

For a VeriSign CA, you must set up a VeriSign ON-SITE account. For information on how to do this, contact VeriSign at www.verisign.com. See “Using VeriSign Certificates” on page 118.

For a Baltimore CA, see “Using Baltimore Certificates” on page 126.

Chapter 252

Installing HP-UX IPSec

Step 2: Loading the HP-UX IPSec Software

Follow the steps below to load HP-UX IPSec software using the HP-UX swinstall program.

1. Log in as root.

2. Insert the HP-UX IPSec disk into the appropriate drive, or locate the directory into which you downloaded the software from HP Software Depot.

3. Run the swinstall program using the command:

swinstall

This opens the Software Selection window and the Specify Source window.

If you need additional information about the Specify Source window, click the Help button on the window.

4. On the Specify Source window, change the Source Host Name if necessary.

Enter the mount point of the drive in the Source Depot Path field and click OK to return to the Software Selection window.

The Software Selection window now contains a list of available software bundles to install.

5. Highlight the HP-UX IPSec software for your system type.

6. Choose Mark for Install from the Actions menu to choose the product to be installed. With the exception of the manpages and user’s manual, you must install the complete IPSec product.

7. Choose Install from the Actions menu to begin product installation and open the Install Analysis window.

8. Click OK in the Install Analysis window when the Status field displays a Ready message.

9. Click Yes in the Confirmation window to confirm that you want to install the software. swinstall displays the Install window.

Read processing data while the software is being installed. When the Status field indicates Ready, the Note window opens.

Chapter 2 53

Installing HP-UX IPSec

Step 2: Loading the HP-UX IPSec Software

swinstall loads the fileset, runs the control scripts for the fileset, and builds the kernel. Estimated time for processing: 3 to 5 minutes.

10. Click OK on the Note window to reboot the system.

The user interface disappears and the system reboots.

11. When the system reboots, check the log files in

/var/adm/sw/swinstall.log and /var/adm/sw/swagent.log to make sure the installation was successful.

NOTE Do not run the HP-UX IPSec product when the system is booted in

single-user mode.

Go to Step 3: Setting the HP-UX IPSec Password.

Chapter 254

Installing HP-UX IPSec

Step 3: Setting the HP-UX IPSec Password

When you install HP-UX IPSec, the HP-UX IPSec password is set to ipsec. You must change the HP-UX IPSec password after installing the product to use the autoboot feature and to load and configure security certificates. HP-UX IPSec uses the password to encrypt certificate files that contain cryptography keys for security certificates, and to control access to the ipsec_mgr security certificate configuration GUI.

To set the password, run the following command:

ipsec_admin -newpasswd

The ipsec_admin utility prompts you to establish the HP-UX IPSec password:

IPSEC_ADMIN: Establishing IPSec password, enter IPSec password:

Enter a password. The password must be at least 15 characters long and can contain spaces.

The ipsec_admin utility will prompt you to enter the new password a second time (IPSEC_ADMIN: Re-enter password to verify). Enter the new password again.

Re-establishing the HP-UX IPSec Password

If you have forgotten the HP-UX IPSec password and are using security certificates for IKE authentication, use the following procedure to re-establish the HP-UX IPSec password:

1. Remove /var/adm/ipsec/.ipsec_info from your system.

2. Revoke any certificates from the Certificate Authority (CA).

3. Re-install the product.

4. Follow the instructions above to set the HP-UX IPSec password.

5. Request and install a new certificate for the local system.

Chapter 2 55

Installing HP-UX IPSec

Step 4: Completing Post-Installation Migration Requirement s

Step 4: Completing Post-Installation Migration Requirements

If you are migrating from a previous version of HP-UX IPSec, complete the post-installation migration procedures in Appendix B, “Post-Installation Migration Instructions” on page 291.

Chapter 256

3 Configuring HP-UX IPSec

Chapter 3 57

Configuring HP-UX IPSec

This chapter describes how to configure HP-UX IPSec, including preshared key configuration. If you are using RSA signature authentication for IKE, you must also see Chapter 4, “Using Certificates with HP-UX IPSec,” on page 113 for instructions on configuring certificates. This chapter also describes how to maximize HP-UX IPSec security and how to use the HP-UX IPSec configuration utility, ipsec_config.

This chapter contains the following sections:

• “Maximizing Security” on page 59

• “Using ipsec_config” on page 61

• “Configuration Overview” on page 66

• “Step 1: Configuring Host IPSec Policies” on page 69

• “Step 2: Configuring Tunnel IPSec Policies” on page 81

• “Step 3: Configuring IKE Policies” on page 89

• “Step 4: Configuring Preshared Keys Using Authentication Records” on page 95

• “Step 5: Configuring Certificates” on page 100

• “Step 6: Configuring the Bypass List (Local IPv4 Addresses)” on page 101

• “Step 7: Verify Batch File Syntax” on page 104

• “Step 8: Committing the Batch File Configuration and Verifying Operation” on page 105

• “Step 9: Configuring HP-UX IPSec to Start Automatically” on page 109

• “Step 10: Creating Backup Copies of the Batch File and Configuration Database” on page 111

Chapter 358

Configuring HP-UX IPSec

Maximizing Security

A system may have both “public” interface IP addresses and “private” interface IP addresses. A public interface IP address is an IP address configured on a Network Interface Card (NIC) connected to a public network. A private interface IP address is an IP address configured on a NIC connected to a private internal network. If you have a system with both a public interface IP address and a private interface IP address, do not assume that all packets processed by the private interface originated from the private network. Do not configure any “open” IPSec policies that allow most or all packets sent to the private interface IP address to pass in clear text.

If you configure an open IPSec policy for a private interface IP address on a system that also has public interfaces, intruders may be able to access services or ports bound to the private interface IP address from other NICs on the system, even if the other interface IP addresses are secured by IPSec policies. Intruders may access services or ports bound to the private interface IP address, even if the intruders are not directly connected to the private interface.

Bypass List

Configuring an entry in the bypass list has the same effect as configuring an open IPSec policy, so the same conditions exist. Intruders may be able to access services or ports bound to the address in the bypass list from other interfaces on the system, even if the other interfaces are secured by IPSec policies. Intruders may access services or ports bound to the address in the bypass list even if the intruders are not directly connected to the interface in the bypass list.

HP recommends that you do not configure open IPSec policies, or entries in the bypass list for private interfaces on systems that also have public interfaces, or on systems on which you are using HP-UX IPSec as a filter or firewall to protect your network.

Strong End System Model

To maximize security when using open policies or the bypass list, HP recommends that you enable the RFC 1122 Strong End-System Model. You can do this by entering the following command:

Chapter 3 59

Configuring HP-UX IPSec

Maximizing Security

ndd -set /dev/ip ip_strong_es_model 1

You can also enable the RFC 1122 Strong End-System model at system startup time by editing the /etc/rc.config.d/nddconf file. Refer to the ndd (1M) manpage for more information.

Chapter 360

Configuring HP-UX IPSec

Using ipsec_config

Using ipsec_con fig

The ipsec_config utility adds, deletes and displays HP-UX IPSec configuration objects stored in the configuration database,

/var/adm/ipsec/config.db. If HP-UX IPSec is active and running, ipsec_config also adds and deletes configuration information in the

runtime policy database. The ipsec_config utility supports the following commands:

• ipsec_config add

• ipsec_config batch

• ipsec_config delete

• ipsec_config show

General Syntax Information

Argument Delimiters

Separate arguments using white spaces (blanks, tabs or new lines).

Line Continuation Character (\)

Use a backslash (\) line continuation character to continue command input on multiple lines.

ipsec_config add

The ipsec_config add command adds objects to the configuration database. For example, the following command adds a host IPSec policy to the configuration database.

ipsec_config add host my_host_policy -source 10.1.1.1 \

-destination 10.0.0.0/8/TELNET -pri 100 \

-action ESP_AES128_HMAC_SHA1

Chapter 3 61

Configuring HP-UX IPSec

Using ipsec_config

ipsec_config batch

The ipsec_config batch command allows you to use ipsec_config in batch mode. In batch mode, ipsec_config reads add and delete operations from a file. Batch mode allows administrators to add and delete multiple configuration objects in one operation. This is useful if you are adding or deleting configuration records that affect other operations.

HP recommends that you use a batch file to add configuration information. A batch file provides a permanent record of the configuration data and can be used to re-create the configuration database.

Batch File Processing

The ipsec_config utility processes the operations in a batch file as a group. If one operation is invalid, all operations in the batch file fail. The ipsec_config utility first verifies each operation in the batch file for syntax errors and collisions (object names and priority values) with existing entries in the configuration database. If all operations in the batch file are valid, the HP-UX IPSec infrastructure updates the configuration database with all operations at the same time. If HP-UX IPSec is active and running, the HP-UX IPSec infrastructure also updates the runtime policy database.

Batch File Syntax

The syntax for add and delete operations in ipsec_config batch files is the same as the syntax for ipsec_config add and ipsec_config delete commands, but without the leading ipsec_config command name. For example, the following entry is a valid add operation for a batch file:

add host my_host_policy -source 10.1.1.1 \

-destination 10.0.0.0/8/TELNET -pri 100 \

-action ESP_AES128_HMAC_SHA1

Comments Lines starting with a pound sign (#) are interpreted as comments. Comment lines within an operation are not allowed.

Chapter 362

Configuring HP-UX IPSec

Using ipsec_config

ipsec_config delete

The ipsec_config delete command deletes objects from the configuration and runtime databases. For example, the following command deletes the host IPSec policy my_host_policy from the configuration database:

ipsec_config delete host my_host_policy

ipsec_config show

The ipsec_config delete command displays objects in the configuration database. For example, the following command displays the host IPSec policies in the configuration database:

ipsec_config show host

The ipsec_config show all command displays the entire contents of the database.

Profile File

An ipsec_config profile file contains default argument values that are evaluated in ipsec_config add commands if the user does not specify the values in the command. The values are evaluated once, when the policy is added to the configuration database. Values used from the profile file become part of the configuration record for the policy.

You can specify a profile file name with the -profile argument as part of an ipsec_config command. By default, ipsec_config uses the /var/adm/ipsec/.ipsec_profile profile file, which is shipped with HP-UX IPSec. In most topologies, you can use the default values supplied in the /var/adm/ipsec/.ipsec_profile file.

HP-UX IPSec also has internal default values that are the same as the values in the /var/adm/ipsec/.ipsec_profile file shipped with the product. If the /var/adm/ipsec/.ipsec_profile file does not exist and the user does not specify an alternate profile file, HP-UX IPSec uses its internal default values.

Chapter 3 63

Configuring HP-UX IPSec

Using ipsec_config

Using a Profile File with a Batch File

The profile argument is illegal inside batch files (you cannot specify the profile argument as part of a statement inside a batch file). You can specify the profile argument as part of the ipsec_config batch command line and ipsec_config will apply it to all entries in the batch file. Refer to the ipsec_config (1M) manpage for more information.

Profile File Structure

The profile file is separated into sections that contain default parameter values for different configuration objects. For example, the HostPolicy-Defaults section contains defaults for host IPSec policies, which are created using the ipsec_config add host command. Each section is delimited by BEGIN and END statements.

Creating a Customized Profile File

In most topologies, you can use the default values in /var/adm/ipsec/.ipsec_profile. If you want to create a customized profile file, make a copy of the /var/adm/ipsec/.ipsec_profile file and edit the copy with a text editor.

You may want to create a customized profile file to change the default source address parameter (source parameter) in the following topologies:

•IPv6 networks

• Multihomed nodes with private interfaces

The default source address parameter values in /var/adm/ipsec/.ipsec_profile are 0.0.0.0/0/0 (IPv4 address

0.0.0.0, address prefix length 0, port 0). This matches any IPv4 address

and any port number. In most topologies, this is appropriates since the default source (local) address will be any IPv4 address on the local system.

IPv6 Networks If you have a network that primarily contains IPv6 addresses, you can change the source parameter value to match any IPv6 address and any port number (0:0/0/0) in the HostPolicy-Defaults, GWPolicy-Defaults, and TunnelPolicy-Defaults sections of the profile file. You can also change the remote parameter value in the IKEPolicy-Defaults section to match any IPv6 address (0::0/0).

Chapter 364

Configuring HP-UX IPSec

Using ipsec_config

Multihomed Nodes with Private Interfaces If the local system is multihomed with one public IP interface and one or more private IP interfaces, you may want to secure only the one public IP interface. In this case, you can set the default source parameter value to the address of the public IP interface in the HostPolicy-Defaults, GWPolicy-Defaults, and TunnelPolicy-Defaults sections of the profile file.

Dynamic Configuration Updates

The ipsec_config utility dynamically updates the configuration database. If HP-UX IPSec is running, ipsec_config also updates the runtime IPSec policy database, and runtime IKE configuration data (IKE policies and authentication records).

Dynamic Deletions

If you delete an object while HP-UX IPSec is running, HP-UX IPSec deletes it from its runtime database. If you delete an IPSec policy, HP-UX IPSec deletes any associated IPSec/QM SAs. If you delete an IKE policy, HP-UX deletes any associated ISAKMP/MM SAs. IPSec/QM SAs negotiated using the ISAKMP/MM SAs may continue to operate, but IKE peers will be unable to send control messages for the affected IPSec/QM SAs.

nocommit Argument

The nocommit argument validates entries but does not update the configuration and runtime databases. The nocommit argument is illegal inside batch files (you cannot specify the nocommit argument as part of a statement inside a batch file). You can specify the nocommit argument as part of the ipsec_config batch command line and ipsec_config will apply it to all entries in the batch file. Refer to the ipsec_config (1M) manpage for more information.

Chapter 3 65

Configuring HP-UX IPSec

Configuration Overvie w

Configuration Overview

There are seven main configuration components:

• Host IPSec Policies

Host IPSec policies specify HP-UX IPSec behavior for IP packets sent or received by the local system as an end host. A host IPSec policy contains address specifications used to select the host IPSec policy for a packet. A host IPSec policy also specifies the HP-UX IPSec behavior (action) for packets using the policy: pass the packets in clear text, discard the packets, or apply an IPSec transform (AH or ESP) to the packets.

• Tunnel IPSec Policies

Tunnel IPSec policies specify the behavior for tunnel endpoints. If the local system is an end host in a host-to-host tunnel topology, or the end host in a host-to-gateway tunnel topology, you must configure tunnel IPSec policies. If the local system is only an end host with no IPSec tunneling, do not configure tunnel IPSec policies.

• IKE Policies

An IKE policy defines the parameters used when negotiating an ISAKMP/Main Mode Security Association (SA). IPSec uses ISAKMP SAs to negotiate IPSec SAs; an ISAKMP SA must exist with a remote system before IPSec can negotiate IPSec SAs.

• IKE Authentication Records

IKE Authentication records contain information that IKE uses to authenticate the identity of the remote system, including local and remote ID values, and preshared keys, if preshared keys are used. You must configure IKE authentication records if you use preshared keys for IKE authentication.

• Security Certificates

You can use security certificates with RSA signatures for IKE authentication (also referred to as primary authentication) instead of preshared keys.

• Bypass List

Chapter 366

Configuring HP-UX IPSec

Configuration Overvie w

The bypass list specifies the local IPv4 addresses that IPSec will bypass or ignore. The system will not attempt to find an IPSec policy for packets sent or received using an IP address in the bypass list, and will process these packets as if HP-UX IPSec was not enabled.

The bypass list improves transmission rates for addresses in the bypass list and is useful in topologies where most of the network traffic passes in clear text and only specific traffic must be secured by IPSec.

The bypass list does not support IPv6 addresses.

• Start-up options

The start-up options allow you to configure HP-UX IPSec to start automatically at system boot-up time and to specify general operating parameters.

HP-UX IPSec also supports gateway IPSec policies when used with HP-UX Mobile IPv6. See “HP-UX IPSec and HP-UX Mobile IPv6” on page 199 for more information on using gateway IPSec policies.

Although you can configure the above components in any order, HP recommends that you use the following procedure to configure IPSec:

Step 1. Configure host IPSec policies.

See “Step 1: Configuring Host IPSec Policies” on page 69 for a description of this step.

Step 2. Configure tunnel IPSec policies.

See “Step 2: Configuring Tunnel IPSec Policies” on page 81 for a description of this step. Skip this step if the local system is not a tunnel endpoint.

Step 3. Configure IKE policies.

See “Step 3: Configuring IKE Policies” on page 89 for a description of this step.Skip this step if the local system uses only manual keys for IPSec.

Step 4. Configure IKE preshared keys using authentication records.

See “Step 4: Configuring Preshared Keys Using Authentication Records” on page 95 for a description of this step. Skip this step if the local system uses only manual keys for IPSec.

Chapter 3 67

Configuring HP-UX IPSec

Configuration Overvie w

Step 5. Configure security certificates and IKE ID information, if you are using

Step 6. Configure the bypass list of local IPv4 addresses (optional).

Step 7. Verify the batch file.

Step 8. Commit the batch file operations to the database and start HP-UX IPSec

RSA signatures for IKE authentication.

See Chapter 4, “Using Certificates with HP-UX IPSec,” on page 113 for a description of this step.

See “Step 6: Configuring the Bypass List (Local IPv4 Addresses)” on page 101 for a description of this step.

HP recommends that you use an ipec_config batch file to add configuration information, and that you use the ipsec_config batch command with the nocommit option to verify the contents of the batch file before committing the batch file operations to the database file. See “Step 7: Verify Batch File Syntax” on page 104 for a description of this step.

to verify operation.

After you have verified the contents of the batch file, commit the batch file operations to the configuration database file. Start HP-UX IPSec and verify operation. See “Step 8: Committing the Batch File Configuration and Verifying Operation” on page 105 for a description of this step.

Step 9. Configure HP-UX IPSec to start automatically at system boot-up time

(optional).

See “Step 9: Configuring HP-UX IPSec to Start Automatically” on page 109 for a description of this step.

Step 10. Back up the ipsec_config batch file and the configuration database.

See “Step 10: Creating Backup Copies of the Batch File and Configuration Database” on page 111 for a description of this step.

Chapter 368

Configuring HP-UX IPSec

Step 1: Configuring Host IPSec Policies

Host IPSec policies specify HP-UX IPSec behavior for IP packets sent or received by the local system as an end host. Each host IPSec policy includes address specifications used to select the host IPSec policy for a packet, and the action for packets using the policy: pass the packets in clear text, discard the packets, or apply an IPSec transform (AH or ESP) to the packets.

If the host policy is for an end host in a host-to-host tunnel topology or an end host in a host-to-gateway topology, the host policy includes a reference to a tunnel policy.

HP recommends that you use an ipsec_config batch file to configure host IPSec policies.

Policy Order and Selection

When an IPSec system sends a packet or receives a packet for an address on the local system, HP-UX IPSec searches the host IPSec policies according to the value of the priority parameter for each policy and selects the first policy with address, protocol and port specifications that match the packet. HP-UX IPSec then takes the action specified in the selected host IPSec policy.

default Host IPSec Policy

The HP-UX IPSec configuration database includes a host IPSec policy named default. HP-UX IPSec uses the default host IPSec policy for a packet if no other host IPSec policies match the packet. The default host IPSec policy allows packets to pass in clear text. You cannot delete the default host IPSec policy, or modify any argument values except the value for the its behavior (the action argument). Use the following command to change the default host IPSec policy so it discards packets:

ipsec_config add host default -action DISCARD

To change back the behavior of the default host IPSec policy to pass packets in clear text, use the following command:

ipsec_config add host default -action PASS

Chapter 3 69

Configuring HP-UX IPSec

Step 1: Configuring Host IPSec Policies

Automatic Priority Increment

You can explicitly set the priority of an IPSec policy with the priority argument, or you can use the automatic priority increment value for host policies in the profile file (the priority parameter value in the

HostPolicy-Defaults section of the profile file). If you do not specify a priority argument, ipsec_config assigns a priority value that is set to

the current highest priority value (lowest priority) in the configuration data base, incremented by the automatic priority increment value for host policies. The result is that the new policy will be the last policy evaluated before the default policy. The default automatic priority increment value (priority) is 10.

If you are configuring the first host IPSec policy and do not specify a priority argument, ipsec_config assigns the automatic priority

increment value as the priority.

ipsec_config add host Syntax

If you are not using manual keys, you can use the following ipsec_config add host syntax in most installations:

ipsec_config add host [-source [-destination [-protocol [-action PASS|DISCARD|

HP recommends that you use an ipsec_config batch file to configure HP-UX IPSec. To specify an add host operation for an ipsec_config batch file, use the above syntax without the ipsec_config command name:

add host [-source [-destination [-protocol [-action PASS|DISCARD|

The full ipsec_config add host syntax specification also allows you to specify the following arguments:

• nocommit (verify the syntax but do not commit the information to the database)

• profile (alternate profile file)

ip_addr[/prefix

ip_addr

protocol_id

host_policy_name ip_addr

[/[

ip_addr[/prefix

protocol_id

host_policy_name

][/

port_number|service_name

[/[

prefix

] [-priority

][/

transform_list

prefix

][/

port_number|service_name

][/

] [-priority

transform_list

port_number|service_name

priority_number

] [-flags

flags

priority_number

] [-flags

flags

]]]

]

]]]

]

Chapter 370

Configuring HP-UX IPSec

Step 1: Configuring Host IPSec Policies

• in and out (inbound and outbound SA information for manual keys)

Refer to the ipsec_config (1M) manpage for full syntax information.

host_policy_name

The

host_policy_name

policy. This name must be unique for each host IPSec policy and is case-sensitive.

Acceptable Values: 1 - 63 characters. Each character must be an ASCII alphanumeric character, hyphen (-), or underscore (_).

The name default is reserved. See “default Host IPSec Policy” on page 69 for more information.

-source and -destination

ip_addr[/prefix[/port_number|service_name

is the user-defined name for the host IPSec

]]

HP-UX IPSec uses the

service_name

HP-UX IPSec uses the address filter to select an IPSec policy for a packet. Specify a local IP address for the source outbound packet, HP-UX IPSec compares the source address filter with the source address fields in the packet, and the destination address filter with the destination address fields in the packet. For an inbound packet, HP-UX IPSec compares the source address filter with the destination address fields in the packet, and the destination address filter with the source address fields in the packet.

Default: If you do not specify

service_name

destination parameter in the HostPolicy-Defaults section of the profile

file used. The default value for source and destination is 0.0.0.0/0/0 (match any IPv4 address, any port).

ip_addr

Acceptable Values: An IPv4 address in dotted-decimal notation or an IPv6 address in colon-hexadecimal notation. The IP address type (IPv4 or IPv6) must be the same for the source and destination address. HP-UX IPSec does not support unspecified IPv6 addresses. However, you can use the double-colon (::) notation within a specified IPv6 address to denote a number of zeros (0) within an address. The address cannot be a broadcast, subnet broadcast, multicast, or anycast address.

] with the protocol argument to form an address filter.

, ipsec_config uses the value of the source or

The

ip_addr

ip_addr, prefix

is the source or destination IP address.

, and

port_number

ip_addr

, and

port_number

. For an

Chapter 3 71

Configuring HP-UX IPSec

Step 1: Configuring Host IPSec Policies

prefix

that must match when comparing the IP address in a packet with

ip_addr

For IPv4 addresses, a prefix length of 32 bits indicates that all the bits in both addresses must match. This prefix length is equivalent to an address mask of 255.255.255.255. Use a value less than 32 to specify a subnet address filter.

For IPv6 addresses, a prefix length of 128 bits indicates that all the bits in both addresses must match. Use a value less than 128 to specify a subnet address filter.

Range: 0 - 32 for an IPv4 address; 0 - 128 for an IPv6 address. If you are using manual keys, prefix must be 32 if 128 if

Default: 32 if non-zero IPv6 address, or 0 (match any address) if all-zeros address (0.0.0.0 or 0::0). You must specify a prefix value if you specify a port or service name as part of the address filter.

port

Specify the upper-layer protocol with the protocol argument described below.

Acceptable Values: 0 - 65535. 0 indicates all ports. The upper-layer protocol must be TCP or UDP if you specify a non-zero port number.

The

ip_addr

The

prefix

is an IPv6 address.

ip_addr

port

is the prefix length, or the number of leading bits

ip_addr

is a non-zero IPv4 address, 128 if

is the upper-layer protocol (TCP or UDP) port number

is an IPv4 address or

ip_addr

is an

is a

Default: 0 (all ports).

service_name

network service. The ipsec_config utility will add a policy to the configuration database with the appropriate port number and protocol, as listed below. You cannot specify same policy.

Tab l e 3-1 ipsec_config Service Names

The

service_name

is a character string that specifies a

service_name

Service

Name

DNS-TCP 53 TCP DNS-UDP 53 UDP FTP-DATA 20 TCP

Port Protocol

and protocol in the

Chapter 372

Configuring HP-UX IPSec

Step 1: Configuring Host IPSec Policies

Tab l e 3-1 ipsec_config Service Names (Continued)

-protocol

FTP-CONTROL 21 TCP HTTP-TCP 80 TCP HTTP-UDP 80 UDP NTP 123 UDP REXEC 512 TCP RLOGIN 513 TCP RWHO 513 UDP REMSH 514 TCP REMPRINT 515 TCP SMTP 25 TCP TELNET 23 TCP TFTP 69 UDP

protocol_id

Service

Name

Port Protocol

The

protocol_id

HP-UX IPSec uses in the address filter to select an IPSec policy for a packet. You cannot specify protocol and a policy.

Specifying ICMPV6 affects only the following ICMPv6 messages: Echo Request, Echo Reply, Mobile Prefix Solicitation, Mobile Prefix Advertisement.

To ensure proper operation of IPv6 networks, HP-UX IPSec always allows all ICMPv6 messages not listed above to pass in clear text

Acceptable Values: Integer value 0 (any protocol) - 255, or one of the following protocol names:

TCP UDP

Chapter 3 73

is the value or name of the upper-layer protocol that

service_name

in the same

Configuring HP-UX IPSec

Step 1: Configuring Host IPSec Policies

ICMP ICMPV6 IGMP MH (Mobile IPv6 Mobility Headers) ALL (any protocol)

The protocols ICMP and IGMP are valid with IPv4 addresses only. The protocols ICMPV6 and MH are valid with IPv6 addresses only.

The

protocol_id

Default: ALL.

CAUTION Discarding or requiring ICMP messages (Internet Control Message

Protocol messages for IPv4; protocol value 1) to be encrypted or authenticated may cause connectivity problems. See Appendix A, “IPv4 ICMP Messages” on page 282 for more information.

must be TCP or UDP if

port

is non-zero.

-priority

The selecting a host IPSec policy (a lower priority value has a higher priority). The priority must be unique for each host IPSec policy.

Range: 1 - 2147483647. Default: If you do not specify a priority, ipsec_config assigns a priority

value that is set to the current highest priority value (lowest priority) for host IPSec policies in the configuration data base, incremented by the automatic priority increment value (priority) for host IPSec policies specified in the HostPolicy-Defaults section of the profile file (this policy will be the last policy evaluated before the default policy). The default automatic priority increment value (priority) is 10.

If this is the first host IPSec policy created, ipsec_config uses the automatic priority increment value as the priority.

-tunnel

If packets using this host IPSec policy will be tunneled and the local system is one of the tunnel endpoints, use the tunnel argument to specify the use with this host IPSec policy.

priority_number

tunnel_policy_name

is the priority value HP-UX IPSec uses when

, the name of the tunnel IPSec policy to

Chapter 374

Configuring HP-UX IPSec

Step 1: Configuring Host IPSec Policies

-action

The action argument specifies the action HP-UX IPSec will perform on packets using this policy. The action must be PASS (pass in clear text) if this is an end system in a host-to-host tunnel topology.

Default: The action defined for the action parameter in the HostPolicy-Defaults section of the profile file used. The default definition for action is DISCARD.

PASS Allow packets using this host IPSec policy to pass in clear text with no alteration. The default host IPSec policy shipped with the product specifies -action PASS.

DISCARD Discard packets using this host IPSec policy.

transform_list

encryption applied to packets using AH (Authentication Header) and ESP (Encapsulation Security Payload) headers. A transform list specifies the transforms acceptable for packets using the policy. The HP-UX IPSec IKE daemon proposes the transform list when negotiating the transform for IPSec Security Associations (SAs) with a remote system.

The transform list in a host policy are transport transforms and are applicable to the host-to-host SA (end-to-end or transport SA) between the source and destination addresses.

If you are using dynamic keys, the transform list can contain:

• A list that contains up to 2 AH transforms

• A list that contains up to 8 ESP transforms, including Authenticated ESP transforms

• A list that contains one nested AH and ESP transform (ESP nested inside of AH)

Use a comma to separate multiple transform specifications.

The order of transforms in the transform list is significant. The first transform is the most preferable and the last transform is the least preferable. At least one transform must match a transform configured on the remote system.

The format for each transform is:

A transform specifies the IPSec authentication and

transform_name[/lifetime_seconds[/lifetime_kbytes

Chapter 3 75

]]

Configuring HP-UX IPSec

Step 1: Configuring Host IPSec Policies

Where:

transform_name

The

transform_name

Header) or ESP (Encapsulation Security Payload) transform specifications, or a nested AH and ESP transform formed by joining an AH transform and an ESP transform with a plus sign (+). For example, AH_MD5+ESP_3DES.

TIP AES128 is the most secure form of encryption, with performance

comparable to or better than DES and 3DES. For added security, use AES in an authenticated ESP transform, such as ESP_AES128_HMAC_SHA1.

ESP transforms without authentication (such as ESP-AES128) do not provide data integrity and should not be used.

Tab l e 3-2 ipsec_config Transforms

is one of the following AH (Authentication

Transform Name Description

AH_MD5 AH, with 128-bit key Hashed Message

Authentication Code using RSA Message Digest-5, HMAC-MD5.

AH_SHA1 AH, with 160-bit key HMAC using Secure

Hash Algorithm-1, HMAC-SHA1.

ESP_AES128 ESP with 128-bit Advanced Encryption

Standard CBC.

ESP_AES128_HMAC_MD5 ESP AES128, authenticated with

HMAC-MD5.

ESP_AES128_HMAC_SHA1 ESP AES128, authenticated with

HMAC-SHA1.

ESP_DES ESP with 56-bit Data Encryption

Standard, Cipher Block Chaining Mode, DES-CBC.

ESP_DES_HMAC_MD5 ESP DES, authenticated with

HMAC-MD5.

Chapter 376

Step 1: Configuring Host IPSec Policies

Tab l e 3-2 ipsec_config Transforms (Continued)

Transform Name Description

ESP_DES_HMAC_SHA1 ESP DES, authenticated with

HMAC-SHA1.

ESP_3DES ESP with triple-DES CBC, three

encryption iterations, each with a different 56-bit key, 3DES-CBC.

ESP_3DES_HMAC_MD5 ESP 3DES, authenticated with

HMAC-MD5.

ESP_3DES_HMAC_SHA1 ESP 3DES, authenticated with

HMAC-SHA1.

ESP_NULL_HMAC_MD5 ESP with null encryption and

authenticated with HMAC-MD5.

ESP_NULL_HMAC_SHA1 ESP with null encryption and

authenticated with HMAC-SHA1.

Configuring HP-UX IPSec

lifetime_seconds

the IPSec SA, in seconds. A transform lifetime can be specified by time (seconds), and by kilobytes transmitted or received. HP-UX IPSec considers the lifetime to be exceeded if either value is exceeded.

Range: 0 (infinite), or 600 - 4294967294 seconds (approximately 497102 days).

Default: 28,800 (8 hours).

lifetime_kbytes

the IPSec SA, measured by kilobytes transmitted or received. A transform lifetime can be specified by time (seconds), and by kilobytes transmitted or received. HP-UX IPSec considers the lifetime to be exceeded if either value is exceeded.

Range: 0 (infinite), or 5120 - 4294967294 kilobytes.

Default: 0 (infinite).

Chapter 3 77

The

lifetime_seconds

The

lifetime_kbytes

is the maximum lifetime for

Configuring HP-UX IPSec

Step 1: Configuring Host IPSec Policies

CAUTION HP recommends that you do not specify an infinite value for

lifetime_seconds

(0) with a finite value for

lifetime_kbytes

-flags

The a plus sign (+).

Tab l e 3-3 ipsec_config add host Flags

EXCLUSIVE Specifies session-based keying. Session-based

MIPV6 Specifies that this IPSec policy is used for Mobile

flags

are additional options for this policy. Join multiple flags with

Flag Description

keying uses a different pair of IPSec/QM SAs per connection or session. Only packets with the same source IP address, destination IP address, network protocol, source port, and destination port will use the same IPSec/QM SA. Session-based keying incurs more overhead but provides more security and privacy. If you do not specify session-based keying, all packets using the same IPSec policy to the same remote node will share the same IPSec/QM SA pair and cryptography keys.

You cannot specify the EXCLUSIVE flag if you are using manual keys, or the action is PASS or DISCARD.

IPv6 packets. HP-UX IPSec checks the Mobile IPv6 binding cache for routing information. (This flag does not specify or affect any protocol specification for the source or destination address filter used when selecting the IPSec policy for a packet.)

If you specify the MIPV6 flag, you must use manual keys (use -in and -out arguments).

You cannot specify the MIPV6 flag with IPv4 addresses in the source and destination arguments.

NONE No flags.

Chapter 378

Configuring HP-UX IPSec

Step 1: Configuring Host IPSec Policies

Default: The value of the flags parameter in the HostPolicy-Defaults section of the profile file used. The default flags value is NONE.

Host IPSec Policy Configuration Examples

The following batch file entry configures a host IPSec policy that requires all traffic between 10.1.1.1 (the local system) and 10.5.5.5 to use ESP with AES128 encryption and HMAC SHA-1 authentication:

add host apple_banana -source 10.1.1.1 \

-destination 10.5.5.5 -pri 20 -action ESP_AES128_HMAC_SHA1

The following batch file entry configures a host IPSec policy that requires all outbound IPv4 rlogin sessions (where the local system is an rlogin client) to use authenticated ESP, with AES128 encryption and HMAC SHA-1 authentication. The user does not specify the source argument, and the ipsec_config program uses the default source argument value from the /var/adm/ipsec/.ipsec_profile file (0.0.0.0/0/0 - the wildcard IPv4 address and any port). The destination argument specifies the wildcard IPv4 address (0.0.0.0/0), and service name RLOGIN (port 513, protocol TCP).

add host rlogin_out -destination 0.0.0.0/0/RLOGIN \

-pri 100 -action ESP_AES128_HMAC_SHA1

The following batch file entry configures a host IPSec policy that requires telnet requests (where the local system is the telnet server) from subnet 10.0.0.0 to use authenticated ESP, with AES128 encryption and HMAC SHA-1 authentication.

add host telnet_in -source 0.0.0.0/0/TELNET \

-destination 10.0.0.0/8 -pri 120 \

-action ESP_AES128_HMAC_SHA1

The following batch file entry configures a host IPSec policy for an application that listens for requests on local TCP port 50000. The policy requires all packets connecting to the application to use AH with HMAC SHA-1 authentication.

add host my_app -source 0.0.0.0.0/0/50000 -protocol TCP \

-pri 140 -action AH_SHA1

The local system (10.1.1.1) is using a host-to-host tunnel with system

10.2.2.2. The following batch file entry configures a host IPSec policy

that references the tunnel policy my_host_host_tunnel and specifies

Chapter 3 79

Configuring HP-UX IPSec

Step 1: Configuring Host IPSec Policies

clear text (no transform) for the transport. See “Tunnel IPSec Policy Configuration Example” on page 87 for the batch file entry used to configure the tunnel IPSec policy my_host_host_tunnel.

The priority is 30 to ensure that HP-UX IPSec selects this policy instead of the policies for telnet and the TCP port 50000 application when the local system is communicating with 10.2.2.2.

add host to_orange -source 10.1.1.1 \

-destination 10.2.2.2 -pri 30 -tunnel my_host_host_tunnel \

-action PASS

Chapter 380

Configuring HP-UX IPSec

Step 2: Configuring Tunnel IPSec Poli cies

Step 2: Configuring Tunnel IPSec Policies

Complete this step only if you are using IPSec tunnels. If you are not using IPSec tunnels, continue to “Step 3: Configuring IKE Policies” on page 89.

Tunnel IPSec policies specify HP-UX IPSec behavior for IP packets tunneled by the local system. In an IPSec tunnel, a tunnel endpoint system encapsulates the original packet in a new IPSec packet with an AH or ESP header. The other tunnel endpoint system processes the AH or ESP header, decapsulates the packet, and sends the packet to the destination address in the original packet header.

An HP-UX system can be the end host in a host-to-host tunnel topology, or the end host in a host-to-gateway tunnel topology.

If the system is an HP-UX Mobile IPv6 Home Agent, it can also act as a gateway, but only when forwarding packets between a Mobile IPv6 client and its Correspondent Node. See “HP-UX IPSec and HP-UX Mobile IPv6” on page 199 if you are configuring HP-UX IPSec for Mobile IPv6.

Tunnel IPSec policies are referenced in host or gateway IPSec policies. HP-UX IPSec first selects a host or gateway IPSec policy to use for a packet. If the IPSec policy specifies a tunnel policy, HP-UX IPSec uses the information in the tunnel IPSec policy to establish an IPSec tunnel with the tunnel destination.

If the local system is a tunnel endpoint, you must configure tunnel IPSec policies. HP recommends that you use an ipsec_config batch file to configure tunnel IPSec policies.

ipsec_config add tunnel Syntax

If you are not using manual keys, you can use the following ipsec_config add tunnel syntax in most installations:

ipsec_config add tunnel

-tsource

[-source [-destination [-protocol

Chapter 3 81

tunnel_address ip_addr[/prefix

ip_addr[/prefix

protocol_id

tunnel_policy_name

-tdestination ][/

port_number|service_name

][/

port_number|service_name

] [-action

transform_list

tunnel_address

]

]]]

Configuring HP-UX IPSec

Step 2: Configuring Tunnel IPSec Poli cies

HP recommends that you use an ipsec_config batch file to configure HP-UX IPSec. To specify an add tunnel operation for an ipsec_config batch file, use the above syntax without the ipsec_config command name:

add tunnel

-tdestination [-source [-destination [-protocol

The full ipsec_config add tunnel syntax specification also allows you to specify the following arguments:

• nocommit (verify the syntax but do not commit the information to the

database)

• profile (alternate profile file)

• in and out (inbound and outbound SA information for manual keys)

Refer to the ipsec_config (1M) manpage for full syntax information.

tunnel_policy_name

tunnel_address

ip_addr[/prefix

protocol_id

] [-action

-tsource

][/

port_number|service_name

][/

port_number|service_name

tunnel_address

transform_list

]]]

]

tunnel_policy_name

The

tunnel_policy_name

policy. This name must be unique for each tunnel IPSec policy and is case-sensitive.

Acceptable Values: 1 - 63 characters. Each character must be an ASCII alphanumeric character, hyphen (-), or underscore (_).

-tsource and -tdestination

is the user-defined name for the tunnel IPSec

tunnel_address

The

tunnel_address

-tsource

-tdestination

Acceptable Values: An IPv4 address in dotted-decimal notation or an IPv6 address in colon-hexadecimal notation. The IP address type (IPv4 or IPv6) must be the same for the tunnel source and destination address. HP-UX IPSec does not support unspecified IPv6 addresses. However, you can use the double-colon (::) notation within a specified IPv6 address to denote a number of zeros (0) within an address. The address must be a unicast address.

Default: None.

tunnel_address

is the IP address for the tunnel endpoint. The

is the local tunnel endpoint; the

tunnel_address

is the remote tunnel endpoint.

Chapter 382

Step 2: Configuring Tunnel IPSec Poli cies

-source and -destination

ip_addr[/prefix[/port_number|service_name

Configuring HP-UX IPSec

]]

HP-UX IPSec uses the

service_name

identifier. When negotiating an outbound IPSec tunnel SA, HP-UX IPSec uses the source address identifier as the proxy source ID, and uses the destination address identifier as the proxy destination ID. When negotiating an inbound IPSec tunnel SA, HP-UX IPSec uses the destination address identifier as the proxy source ID and the source address identifier as the proxy destination ID. The proxy ID values must exactly match the proxy ID values on the remote system.

If you are using manual keys with an IPv6 ESP, HP-UX IPSec also uses the address identifier to verify the address fields in the original (end-to-end) packet. For an outbound tunneled packet (the local address is the source address in the tunnel packet header), HP-UX IPSec verifies the source address identifier with the source address fields in the original packet, and the destination address identifier with the destination address fields in the original packet. For an inbound tunneled packet (the local address is the destination address in the tunnel packet header), HP-UX IPSec verifies the source address identifier with the destination address fields in the original packet, and the destination address identifier with the source address fields in the original packet.

Default: If you do not specify

service_name

destination parameter in the TunnelPolicy-Defaults section of the profile file used.of the profile file used. The default value for source and destination is 0.0.0.0/0/0 (match any IPv4 address, any port).

] with the protocol argument to form an address

, ipsec_config uses the value of the source or

ip_addr, prefix

, and

port_number

, and

port_number

Where:

ip_addr

address.

Chapter 3 83

The

ip_addr

is the proxy (end system) source or destination IP

Configuring HP-UX IPSec

Step 2: Configuring Tunnel IPSec Poli cies

prefix

that must match when comparing the IP address in a packet with

ip_addr

For IPv6 addresses, a prefix length of 128 bits indicates that all the bits in both addresses must match. Use a value less than 128 to specify a subnet address filter.

Range: 0 - 32 for an IPv4 address; 0 - 128 for an IPv6 address. If you are using manual keys, prefix must be 32 if 128 if

port

Specify the upper-layer protocol with the protocol argument described below. The upper-layer protocol must be TCP or UDP if you specify a non-zero port number

The

ip_addr

The

prefix

is an IPv6 address.

ip_addr

port

is the prefix length, or the number of leading bits

ip_addr

is a non-zero IPv4 address, 128 if

is the upper-layer protocol (TCP or UDP) port number

is an IPv4 address or

ip_addr

is an

is a

Acceptable Values: 0 - 65535. 0 indicates all ports.

NOTE The port value must be 0 if the corresponding host policy (the host policy

that references this tunnel policy) uses a transform (the corresponding host policy action is not PASS).

Default: 0 (all ports).

service_name

network service. The ipsec_config utility will add a policy to the configuration database with the appropriate port number and protocol, as listed below. You cannot specify same policy.

See Table 3-1, “ipsec_config Service Names,” on page 72 for a list of valid service names.

The

service_name

is a character string that specifies a

service_name

and protocol in the

Chapter 384

Configuring HP-UX IPSec

Step 2: Configuring Tunnel IPSec Poli cies

-protocol

The

protocol

HP-UX IPSec uses in the address filter to select an IPSec policy for a packet. You cannot specify protocol and policy.

Specifying ICMPV6 affects only the following ICMPv6 messages: Echo Request, Echo Reply, Mobile Prefix Solicitation, Mobile Prefix Advertisement.

To ensure proper operation of IPv6 networks, HP-UX IPSec always allows all ICMPv6 messages not listed above to pass in clear text

Acceptable Values: Integer value 0 (any protocol) - 255, or one of the following protocol names:

TCP UDP ICMP ICMPV6 IGMP MH (Mobile IPv6 Mobility Headers) ALL (any protocol)

The protocols ICMP and IGMP are valid with IPv4 addresses only. The protocols ICMPV6 and MH are valid with IPv6 addresses only.

protocol_id

is the value or name of the upper-layer protocol that

service_name

in the same

NOTE The protocol value must be AL L or 0 if the corresponding host policy (the

host policy that references this tunnel policy) uses a transform (the host policy action is not PASS).

Default: ALL.

CAUTION Discarding or requiring ICMP messages (Internet Control Message

Protocol messages for IPv4; protocol value 1) to be encrypted or authenticated may cause connectivity problems. See Appendix A, “IPv4 ICMP Messages” on page 282 for more information.

Chapter 3 85

Configuring HP-UX IPSec

Step 2: Configuring Tunnel IPSec Poli cies

-action

A transform specifies the IPSec authentication and encryption applied to packets using AH (Authentication Header) and ESP (Encapsulation Security Payload) headers. A transform list specifies the transforms acceptable for packets using the policy. The HP-UX IPSec IKE daemon proposes the transform list when negotiating the transform for IPSec Security Associations (SAs) with a remote system.

The packets encapsulated between the tunnel endpoints.

If you are using dynamic keys, the transform list can contain:

• A list that contains up to 2 AH transforms

• A list that contains up to 8 ESP transforms, including Authenticated

• A list that contains one nested transform (ESP nested inside of AH)

Use a comma to separate multiple transform specifications.

transform_list

ESP transforms

transform

in a tunnel policy are tunnel transports applied to

The format for each transform is:

transform_name[/

Where:

lifetime_seconds[/lifetime_kbytes

]]

transform_name

(Encapsulation Security Payload) transform name, as specified in Table 3-2, “ipsec_config Transforms,” on page 76, or a nested AH and ESP transform formed by joining an AH transform and an ESP transform with a plus sign (+). For example, AH_MD5+ESP_3DES.

TIP AES128 is the most secure form of encryption, with performance

comparable to or better than DES and 3DES. For added security, use AES in an authenticated ESP transform, such as ESP_AES128_HMAC_SHA1.

is a valid AH (Authentication Header) or ESP

Chapter 386

Configuring HP-UX IPSec

Step 2: Configuring Tunnel IPSec Poli cies

ESP transforms without authentication (such as ESP-AES128) do not provide data integrity and should not be used.

Default: The transform defined for the action parameter in the TunnelPolicy-Defaults section of the profile file used. The default action is ESP_AES128_HMAC_SHA1.

lifetime_seconds

The

lifetime_seconds

seconds. A transform lifetime can be specified by time (seconds), and by kilobytes transmitted or received. HP-UX IPSec considers the lifetime to be exceeded if either value is exceeded.

Range: 0 (infinite), or 600 - 4294967294 seconds (approximately 497102 days).

Default: 28,800 (8 hours).

is the maximum lifetime for the IPSec SA, in

lifetime_kbytes

The

lifetime_kbytes

measured by kilobytes transmitted or received. A transform lifetime can be specified by time (seconds), and by kilobytes transmitted or received. HP-UX IPSec considers the lifetime to be exceeded if either value is exceeded.

is the maximum lifetime for the IPSec SA,

Range: 0 (infinite), or 5120 - 4294967294 kilobytes.

Default: 0 (infinite).

CAUTION HP recommends that you do not specify an infinite value for

lifetime_seconds

(0) with a finite value for

lifetime_kbytes

Tunnel IPSec Policy Configuration Example

The local system (10.1.1.1) is using a host-to-host tunnel with system

10.2.2.2. The following batch file entry configures the tunnel to use authenticated ESP, with AES128 encryption and HMAC SHA-1 authentication.

Chapter 3 87

Configuring HP-UX IPSec

Step 2: Configuring Tunnel IPSec Poli cies

ipsec_config add tunnel my_host_host_tunnel \

-tsource 10.1.1.1 -tdestination 10.2.2.2 \

-source 10.1.1.1 -destination 10.2.2.2 \

-action ESP_AES128_HMAC_SHA1

Chapter 388

Configuring HP-UX IPSec

Step 3: Confi guring IKE Policies

Step 3: Configuring IKE Policies

Complete this step only if you are using dynamic keys for IPSec. You do not need to configure IKE policies if you are using only manual keys for IPSec, or if you are only using HP-UX IPSec to discard packets. If you are not using dynamic keys, go to “Step 6: Configuring the Bypass List (Local IPv4 Addresses)” on page 101

HP-UX IPSec uses the parameters in an IKE policy when using the IKE protocol to establish ISAKMP/Main Mode (MM) Security Associations (SAs) with remote systems. IPSec uses ISAKMP/MM SAs to negotiate IPSec SAs; an ISAKMP/MM SA must exist with a remote system before IPSec can negotiate IPSec SAs.

You must have at least one IKE policy if you are using dynamic keys for IPSec. If HP-UX IPSec cannot find an IKE policy with a remote address specification that matches the remote system, the ISAKMP/MM SA negotiation will fail.

HP recommends that you use an ipsec_config batch file to configure IKE policies.

Policy Order and Selection

When HP-UX IPSec needs to establish an ISAKMP/MM SA, it searches the IKE policies according to the value of the priority parameter for each policy and selects the first policy with the IP address and prefix specifications that match the remote system’s address.

Automatic Priority Increment

You can explicitly set the priority of an IKE policy with the priority argument, or you can use the automatic priority increment value for IKE policies in the profile file (the priority parameter value in the

IKEPolicy-Defaults section of the profile file). If you do not specify a priority argument, ipsec_config assigns a priority value that is set to

the current highest priority value (lowest priority) in the configuration data base, incremented by the automatic priority increment value for IKE policies. The result is that the new policy will be the last policy evaluated before the default policy. The default automatic priority increment value (priority) is 10.

Chapter 3 89

Configuring HP-UX IPSec

Step 3: Configuring IKE Policies

If you are configuring the first IKE policy and do not specify a priority argument, ipsec_config assigns the automatic priority increment value

as the priority.

ipsec_config add ike Syntax

You can use the following ipsec_config add ike syntax in most installations:

ipsec_config add ike

-remote [-authentication PSK|RSASIG] [-hash MD5|SHA1] [-encryption DES|3DES] [-life

HP recommends that you use an ipsec_config batch file to configure HP-UX IPSec. To specify an add ike operation for an ipsec_config batch file, use the above syntax without the ipsec_config command name:

add ike

-remote [-authentication PSK|RSASIG] [-hash MD5|SHA1] [-encryption DES|3DES] [-life

The full ipsec_config add ike syntax specification also allows you to specify the following arguments:

• nocommit (verify the syntax but do not commit the information to the

• profile (alternate profile file)

Refer to the ipsec_config (1M) manpage for full syntax information.

ip_addr[/prefix

lifetime_seconds

ike_policy_name ip_addr[/prefix

lifetime_seconds

database)

ike_policy_name

] [-priority

] [-maxqm|mq

] [-priority

] [-maxqm|mq

priority_number

max_quick_modes

priority_number

max_quick_modes

]

ike_policy_name

The

ike_policy_name

name must be unique for each IKE policy and is case-sensitive.

Acceptable Values: 1 - 63 characters. Each character must be an ASCII alphanumeric character, hyphen (-), or underscore (_).

is the user-defined name for the IKE policy. This

Chapter 390

Configuring HP-UX IPSec

Step 3: Confi guring IKE Policies

-remote

The that specifies the remote system or subnet for this policy. HP recommends that you do not specify a wildcard address (0.0.0.0/0 or 0::0/0). Wildcard addresses allow unauthorized systems to engage the local systems in IKE negotiations.

Where:

ip_addr

Default: None.

prefix

that must match when comparing the remote IP address with

ip_addr[/prefix

ip_addr

The

and

ip_addr

prefix

]

prefix

are the IP address and network prefix length

is the remote IP address.

is the prefix length, or the number of leading bits

ip_addr

For IPv6 addresses, a prefix length of 128 bits indicates that all the bits in both addresses must match. Use a value less than 128 to specify a subnet address filter.

Range: 0 - 32 for an IPv4 address; 0 - 128 for an IPv6 address. If you are using manual keys, prefix must be 32 if 128 if

ip_addr

Default: 32 if non-zero IPv6 address, or 0 (match any address) if

all-zeros address (0.0.0.0 or 0::0).

-priority

The

priority_number

selecting an IKE policy (a lower priority value has a higher priority). The priority must be unique for each IKE policy.

Chapter 3 91

is an IPv6 address.

ip_addr

is a non-zero IPv4 address, 128 if

priority_number

is the priority value HP-UX IPSec uses when

ip_addr

is an IPv4 address or

ip_addr

is an

is a

Configuring HP-UX IPSec

Step 3: Configuring IKE Policies

Range: 1 - 2147483647. Default: If you do not specify a priority, ipsec_config assigns a priority

value that is set to the current highest priority value (lowest priority) for IKE policies in the configuration data base, incremented by the automatic priority increment value (priority) for IKE policies specified in the HostPolicy-Defaults section of the profile file (this policy will be the last policy evaluated before the default policy). The default automatic priority increment value (priority) is 10.

If this is the first IKE policy created, ipsec_config uses the automatic priority increment value as the priority.

-authentication

The

authentication_type

HP-UX IPSec will use when establishing the ISAKMP/MM SA. This must match the method configured on the remote system.

Acceptable Values:

PSK (preshared key) RSASIG (RSA signature using security certificates)

If you specify PSK, you must configure a preshared key using the ipsec_config add auth command. If you specify RSASIG, you must use

security certificates. See Chapter 4, “Using Certificates with HP-UX IPSec,” on page 113 for information on using security certificates with HP-UX IPSec.

Default: The value of the authentication parameter in the IKE-Defaults section of the profile file used. The default authentication parameter value is PSK.

-group 1|2

The group argument specifies the Oakley Group (sometimes referred to as the Diffie-Hellman group) used to select initial Diffie-Hellman values. This must match the Oakley Group configured on the remote system.

Acceptable Values:

authentication_type

is the primary authentication method

1 (MODP, 768-bit exponent) 2 (1024-bit exponent)

Default: The value of the group parameter in the IKE-Defaults section of the profile file used. The default group parameter value is 2.

Chapter 392

Configuring HP-UX IPSec

Step 3: Confi guring IKE Policies

-hash MD5|SHA1

The hash argument specifies the hash algorithm for authenticating IKE messages. This must match the hash algorithm configured on the remote system.

Acceptable Values:

MD5 (128-bit key Hashed Message Authentication Code using RSA Message Digest-5, HMAC-MD5) SHA1 (160-bit key HMAC using Secure Hash Algorithm-1, HMAC-SHA1)

Default: The value of the hash parameter in the IKE-Defaults section of the profile file used. The default hash parameter value is MD5.

-encryption

the encryption algorithm for encrypting IKE messages. This must match the encryption algorithm configured on the remote system.

Acceptable Values:

DES (56-bit Data Encryption Standard, Cipher Block Chaining Mode, DES-CBC) 3DES (triple-DES CBC, three encryption iterations, each with a different 56-bit key, 3DES-CBC)

Default: The value of the encryption parameter in the IKE-Defaults section of the profile file used. The default encryption parameter value is 3DES.

-life

The SA, in seconds.

Range: 0 (infinite), or 600 - 4294967294 seconds (approximately 497102 days).

Default: 28,800 (8 hours).

-maxqm

The (QM) SA negotiations that IKE can perform using an ISAKMP/MM SA. Each IPSec/QM SA negotiation establishes two IPSec SAs (one in each direction).

lifetime_seconds

max_quick_modes

encryption_algorithm

is the maximum lifetime for the ISAKMP/MM

max_quick_modes

is the maximum number of IPSec or Quick Mode

The

encryption_algorithm

Chapter 3 93

Configuring HP-UX IPSec

Step 3: Configuring IKE Policies

If the value of Secrecy (PFS) for the IPSec SA keys and the identities of the ISAKMP negotiating parties (and identities of any parties for which the ISAKAMP parties are acting as proxies). With PFS, the exposure of one key permits access only to data protected by that key. When PFS is configured, the IKE daemon creates a new ISAKMP SA for each IPSec SA negotiation and performs a Diffie-Hellman exchange for each IPSec SA negotiation.

Range: 1 - 255.

Default: 100.

max_quick_modes

is 1, IKE provides Perfect Forward

ipsec_config add IKE Command Examples

The following batch file entries configure two IKE policies. The first policy (apple) is for a remote system (10.1.1.1) that uses RSA signature (security certificate) for IKE authentication. The second policy (all_others) is for all other systems in the local network (10.*.*.*), which use preshared keys for IKE authentication.

The priority argument is omitted, and the automatic priority increment assigns the first policy (apple) a lower priority value (higher priority) than the second policy (all_others).

add ike apple -remote 10.1.1.1 -auth RSASIG add ike all_others -remote 10.0.0.0/8 -auth PSK

Chapter 394

Configuring HP-UX IPSec

Step 4: Configuring Preshared Keys Using Authenticatio n Recor ds

Step 4: Configuring Preshared Keys Using Authentication Records

Complete this step only if you configured PSK (preshared keys) as an IKE authentication method in “Step 3: Configuring IKE Policies” on page 89. If you configured RSASIG (RSA signatures) as the IKE authentication method in all IKE policies, skip this step, and go to Chapter 4, “Using Certificates with HP-UX IPSec,” on page 113.

HP-UX IPSec stores preshared keys in authentication records. You configure authentication records using the ipsec_config add auth command.

Remote Multi-homed Systems

If a remote system is multi-homed (the remote systems has multiple IP addresses), you must configure an authentication record for each IP address on the remote system. Specify the same preshared key in each authentication record for the remote system.

Configuring IKE ID Information with Preshared Keys

Authentication records can also include IKE ID information. You do not have to configure IKE ID information if your topology meets the following requirements:

• you are using preshared keys

• the remote system is an HP-UX IPSec system, or a system that uses IP addresses as IKE IDs

If your topology does not meet the above requirements, you must configure IKE ID information. Refer to the ipsec_config (1M) manpage for more information on configuring IKE ID information. Chapter 4, “Configuring Authentication Records with IKE IDs” on page 134 also contains information on configuring IKE ID information.

As part of the ISAKMP/MM SA negotiation, the IKE peers exchange and verify ID types and ID values. During an ISAKMP/MM negotiation, HP-UX IPSec uses the remote system address to search for an

Chapter 3 95

Configuring HP-UX IPSec

Step 4: Configuring Preshared Ke ys Using Authentication Records

authentication record. For preshared key authentication, the authentication record contains the preshared key value and can also contain the following IKE ID information:

•local ID type

•local ID value

• remote ID type

• remote ID value

If the authentication record matching the remote address includes local ID information, HP-UX IPSec sends the configured local ID information in an ISAKMP ID payload. If the matching authentication record has no local ID information, HP-UX IPSec sends the IP address of the interface it is using for the IKE negotiation as the local ID value, and sends the appropriate address type (IPv4 or IPv6) as the local ID type.

If the matching authentication record has remote ID information, HP-UX IPSec uses it to verify what the remote system sends in the ISAKMP ID payload. If the matching authentication record has no remote ID information for the remote system, HP-UX IPSec verifies that the source IP address from the inbound packet matches the ID value sent by the remote system, and uses the appropriate IP address type as the ID type.

ipsec_config add auth Syntax

You can use the following ipsec_config add auth syntax to configure preshared keys in most installations:

ipsec_config add auth

-remote

HP recommends that you use an ipsec_config batch file to configure HP-UX IPSec. To specify an add auth operation for an ipsec_config batch file, use the above syntax without the ipsec_config command name:

add auth

-remote

The full ipsec_config add auth syntax specification also allows you to specify the following arguments:

ip_addr[/prefix

auth_name

ip_addr[/prefix

auth_name

] [-preshared

preshared_key

]

Chapter 396

HP HP-UX IPSec Administrator's Guide

Specifications and Main Features

Frequently Asked Questions

User Manual

Preface: About This Document

1 HP-UX IPSec Overview

Introduction

Authentication Header (AH)

Transport and Tunnel Modes

Encapsulating Security Payload (ESP)

ESP Encryption

Transport and Tunnel Modes

ESP with Authentication and Encryption

Internet Key Exchange (IKE)

Security Associations (SAs) and IKE Phases

Generating Shared Keys: Diffie-Hellman

IKE Primary Authentication

Re-using Negotiations

IKE Automatic Re-keying

Manual Keys

HP-UX IPSec Topologies

Host-to-Host Topology

Host-to-Gateway Topology

Host-to-Host Tunnel Topology

Gateway-to-Gateway Topology

HP-UX IPSec Configurati on and Management Featur es

2 Installing HP-UX IPSec

HP-UX IPSec Product Requirements

Disk Requirements

Security Certificate Configuration Utility Requirements

Step 1: Veri f y ing HP-UX IPSec Install a t ion and Configuration Prerequisit es

Step 2: Loading the HP-UX IPSec Software

Step 3: Setting the HP-UX IPSec Password

Re-establishing the HP-UX IPSec Password

Step 4: Completing Post-Installation Migration Requirement s

3 Configuring HP-UX IPSec

Maximizing Security

Bypass List

Strong End System Model

Using ipsec_con fig

General Syntax Information

ipsec_config add

ipsec_config batch

ipsec_config delete

ipsec_config show

Profile File

Dynamic Configuration Updates

nocommit Argument

Configuration Overvie w

Step 1: Configuring Host IPSec Policies

Policy Order and Selection

ipsec_config add host Syntax

Host IPSec Policy Configuration Examples

Step 2: Configuring Tunnel IPSec Poli cies

ipsec_config add tunnel

Tunnel IPSec Policy Configuration Example

Step 3: Confi guring IKE Policies

Policy Order and Selection

ipsec_config add ike

ipsec_config add IKE Command Examples

Step 4: Configuring Preshared Keys Using Authenticatio n Recor ds

Remote Multi-homed Systems

Configuring IKE ID Information with Preshared Keys

ipsec_config add auth