IBM Tivoli Remote Control User Manual

Front cover

Implementing IBM Tivoli Remote Control Across Firewalls

Achieve Remote Control without sacrificing security

Guide for TCP/IP ports used and troubleshooting

Set up a secure Remote Control environment based on realistic scenarios

ibm.com/redbooks

Edson Manoel

Francesca Balzi

Sebastien Fardel

Venkata R Reddy

International Technical Support Organization

Implementing IBM Tivoli Remote Control Across Firewalls

April 2003

SG24-6944-00

Note: Before using this information and the product it supports, read the information in “Notices” on page xiii.

First Edition (April 2003)

This edition applies to the following products: IBM Tivoli Remote Control 3.8, IBM Tivoli Management Framework 4.1, and Tivoli Firewall Security Toolbox 1.3.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Tabl es . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix

Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xiv

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

Part 1. Concepts and planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Chapter 1. Remote Control sessions overview . . . . . . . . . . . . . . . . . . . . . . 3

1.1 IBM Tivoli Remote Control overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.1.1 IBM Tivoli Management Framework components . . . . . . . . . . . . . . . . 4

1.1.2 IBM Tivoli Remote Control components . . . . . . . . . . . . . . . . . . . . . . . 7

1.1.3 Tivoli components and communication symbols . . . . . . . . . . . . . . . . . 8

1.1.4 Parent-Child concept. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.1.5 Proxy connection types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.2 IBM Tivoli Remote Control sessions overview . . . . . . . . . . . . . . . . . . . . . 12

1.2.1 Session in a single-TMR environment . . . . . . . . . . . . . . . . . . . . . . . 14

1.2.2 Session in a multi-TMR environment . . . . . . . . . . . . . . . . . . . . . . . . 21

1.2.3 Session using a Remote Control Gateway . . . . . . . . . . . . . . . . . . . . 31

1.2.4 Session using Remote Control Proxies Standalone . . . . . . . . . . . . . 35

1.2.5 Session using Remote Control Proxies in a TFST environment . . . . 45

Chapter 2. Implementation planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

2.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.1.1 Logical design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.1.2 Physical design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

2.1.3 Network considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

2.2 Planning for IBM Tivoli Remote Control Proxy . . . . . . . . . . . . . . . . . . . . . 73

2.3 Implementation planning case study scenario . . . . . . . . . . . . . . . . . . . . . 80

Part 2. Implementation scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

Chapter 3. Implementation scenario: Standalone Proxies . . . . . . . . . . . . 93

3.1 Scenario overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

3.2 Environment description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

3.2.1 Technical infrastructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

3.2.2 Data flow description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

3.3 Scenario installation and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 101

3.3.1 Remote Control Proxy installation. . . . . . . . . . . . . . . . . . . . . . . . . . 101

3.3.2 Remote Control Proxy configuration . . . . . . . . . . . . . . . . . . . . . . . . 104

3.3.3 Firewall configuration table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

3.3.4 Remote Control Proxy startup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Chapter 4. Implementation scenario: Tivoli Firewall Security Toolbox . 115

4.1 Scenario overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

4.2 Environment description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

4.2.1 Technical infrastructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

4.2.2 Data flow description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

4.2.3 Firewall configuration tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

4.3 Scenario installation and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 125

4.3.1 Remote Control Proxy installation. . . . . . . . . . . . . . . . . . . . . . . . . . 125

4.3.2 Remote Control Proxy configuration . . . . . . . . . . . . . . . . . . . . . . . . 129

4.3.3 Remote Control Proxy startup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Part 3. Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Chapter 5. Troubleshooting techniques . . . . . . . . . . . . . . . . . . . . . . . . . . 141

5.1 Generic problem determination outline . . . . . . . . . . . . . . . . . . . . . . . . . . 142

5.1.1 Session startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

5.1.2 Session management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

5.2 Troubleshooting the Remote Control Proxy . . . . . . . . . . . . . . . . . . . . . . 155

5.2.1 The rcproxy.log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

5.2.2 The remcon.trc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

5.3 Troubleshooting examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

5.3.1 Case 1: Controller not connecting to Target Proxy . . . . . . . . . . . . . 160

5.3.2 Case 2: Target Proxy service is not active . . . . . . . . . . . . . . . . . . . 163

5.3.3 Case 3: Wrong Proxy configuration . . . . . . . . . . . . . . . . . . . . . . . . 167

5.4 Troubleshooting the firewall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

Part 4. Appendixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Appendix A. Tivoli Firewall Security Toolbox overview. . . . . . . . . . . . . . 175

Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Components of TFST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Endpoint Proxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Gateway Proxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

iv IBM Tivoli Remote Control Across Firewalls

Relay. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Event Sink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

Tivoli environments with single firewall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

Tivoli environments with multiple firewalls . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

Sending events across firewalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

Installation and configuration of TFST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

Installation of TFST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

Configuration of TFST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

TFST components and operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Port range configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Appendix B. Introducing firewalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

Functionality of a firewall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

Firewall tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

Packet filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

Proxy servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

Socks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

DNS and mail gateways . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

Network address translation (NAT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

Virtual Private Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

Log management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

Firewalls in the market . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Contents v

vi IBM Tivoli Remote Control Across Firewalls

Figures

1-1 Parent-Child hierarchy in RC Proxy-TFST architecture . . . . . . . . . . . . . 11

1-2 RC session data flow in a single-TMR environment . . . . . . . . . . . . . . . 14

1-3 RC session data flow in a multi-TMR environment . . . . . . . . . . . . . . . . 22

1-4 RC session data flow in an RC Gateway/single-TMR environment. . . . 32

1-5 RC session data flow in an RC Gateway/multi-TMR environment. . . . . 34

1-6 RC session data flow in an RC Proxy Standalone/single-TMR . . . . . . . 36

1-7 RC session data flow in an RC Proxy Standalone/multi-TMR . . . . . . . . 40

1-8 RC session data flow in an RC Proxy-TFST/single-TMR environment . 46 1-9 RC session data flow in an RC Proxy-TFST/multi-TMR environment . . 51

2-1 Planning overview for RC Proxy in a Standalone environment . . . . . . . 74

2-2 Planning overview for Remote Control Proxy in a TFST environment. . 75

2-3 Case study scenario without RC Proxy architecture . . . . . . . . . . . . . . . 82

2-4 Case study scenario with RC Proxy architecture - Solution A . . . . . . . . 84

2-5 Case study scenario with RC Proxy architecture - Solution B . . . . . . . . 88

3-1 General testing scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

3-2 RC Proxy Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

3-3 Remote Control Data Flow Overview. . . . . . . . . . . . . . . . . . . . . . . . . . 100

3-4 Remote Control Data Flow Overview. . . . . . . . . . . . . . . . . . . . . . . . . . 108

3-5 Startup of Remote Control Proxy using Service Applet . . . . . . . . . . . . 110

4-1 General TFST testing scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

4-2 Remote Control Proxy Implementation in a TFST environment . . . . . 120

4-3 Data flow overview: Non-Standalone scenario . . . . . . . . . . . . . . . . . . 122

5-1 Endpoint problem determination flow. . . . . . . . . . . . . . . . . . . . . . . . . . 144

5-2 TFST problem determination flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

5-3 RC Proxy problem determination flow . . . . . . . . . . . . . . . . . . . . . . . . . 152

5-4 Error message displayed on Controller when attempt a session. . . . . 160

5-5 Error message displayed on the Controller at session startup . . . . . . 163

5-6 Error message at session startup: Proxy configuration problem . . . . . 167

A-1 Tivoli Endpoint and Gateway proxies communication through firewall 177 A-2 Relay connecting Endpoint and Gateway proxies through a DMZ . . . 178

A-3 Event Sink collecting non-TME events . . . . . . . . . . . . . . . . . . . . . . . . 179

viii IBM Tivoli Remote Control Across Firewalls

Tables

2-1 RC ports for unidirectional communication - Parent/initiator . . . . . . . . . 63

2-2 RC ports for unidirectional communication - Parent/listener . . . . . . . . . 64

2-3 RC ports for unidirectional communication - Relay - Parents/initiators . 65 2-4 RC ports for unidirectional communication - Relay - Parents/listeners . 67

2-5 RC ports for bidirectional communication . . . . . . . . . . . . . . . . . . . . . . . 69

2-6 RC ports for bidirectional communication with Relay. . . . . . . . . . . . . . . 71

2-7 Hardware requirements for IBM Tivoli Remote Control Proxy . . . . . . . 76

2-8 Software requirements for IBM Tivoli Remote Control Proxy. . . . . . . . . 77

2-9 RC Proxy network ports for firewall 1 - Solution A. . . . . . . . . . . . . . . . . 86

2-10 RC Proxy network ports for firewall 2 - Solution A . . . . . . . . . . . . . . . . . 87

2-11 RC Proxy network ports for firewall 3 - Solution A . . . . . . . . . . . . . . . . . 87

2-12 RC Proxy network ports for firewall 1 - Solution B . . . . . . . . . . . . . . . . . 90

2-13 RC Proxy network ports for firewall 2 - Solution B . . . . . . . . . . . . . . . . . 90

2-14 RC Proxy network ports for firewall 3 - Solution B . . . . . . . . . . . . . . . . . 90

3-1 RC Target Proxy settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

3-2 RC Controller Proxy settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

3-3 Scenario firewall configuration table . . . . . . . . . . . . . . . . . . . . . . . . . . 108

4-1 Summary of Framework and Remote Control resources . . . . . . . . . . 121

4-2 Summary of port configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

4-3 Firewall 1 configuration table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

4-4 Firewall 2 configuration table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

4-5 RC Target Proxy installation settings . . . . . . . . . . . . . . . . . . . . . . . . . . 126

4-6 Relay instance installation settings . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

4-7 RC Controller Proxy installation settings . . . . . . . . . . . . . . . . . . . . . . . 128

x IBM Tivoli Remote Control Across Firewalls

Examples

1-1 RC session trace in a single-TMR environment. . . . . . . . . . . . . . . . . . . 16

1-2 The remote_control method from a single-TMR environment . . . . . . . . 19

1-3 The is_proxied_ep method from a single-TMR environment . . . . . . . . . 20

1-4 The nd_start_target method from a single-TMR environment . . . . . . . . 20

1-5 The nd_start_controller method from a single-TMR environment . . . . . 20

1-6 RC session trace from the HUB TMR in a multi-TMR environment . . . . 25

1-7 RC session trace from the Spoke TMR in a multi-TMR environment . . 25

1-8 The remote_control method from a Spoke TMR . . . . . . . . . . . . . . . . . . 28

1-9 The is_proxied_ep method from a Spoke TMR . . . . . . . . . . . . . . . . . . . 29

1-10 The nd_start_target method from a Spoke TMR . . . . . . . . . . . . . . . . . . 29

1-11 The nd_start_controller method from a Spoke TMR . . . . . . . . . . . . . . . 29

1-12 The nd_start_controller method from a HUB TMR . . . . . . . . . . . . . . . . 30

1-13 The rc_def_gw default policy method for Remote Control . . . . . . . . . . . 33

1-14 The rc_def_proxy default policy method for Remote Control. . . . . . . . . 38

1-15 The is_proxied_ep method for an RC Proxy Standalone architecture . . 43 1-16 The nd_start_target method for an RC Proxy Standalone architecture . 43 1-17 The nd_start_controller method for RC Proxy Standalone architecture 44

1-18 The rc_def_proxy default policy method for Remote Control. . . . . . . . . 49

1-19 The is_proxied_ep method for an RC Proxy-TFST architecture . . . . . . 54

1-20 The nd_start_target method for an RC Proxy-TFST architecture . . . . . 55

1-21 The nd_start_controller method for an RC Proxy-TFST architecture. . . 55

3-1 Hub TMR region. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3-2 Spoke TMR region. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3-3 Hub TMR Endpoint Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

3-4 Spoke TMR Endpoint Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

3-5 Remote Control Target Proxy configuration file. . . . . . . . . . . . . . . . . . 104

3-6 Remote Control Controller Proxy configuration file . . . . . . . . . . . . . . . 105

3-7 Target Proxy local-port-rage definition. . . . . . . . . . . . . . . . . . . . . . . . . 105

3-8 Remote Control Proxy routing table configuration file . . . . . . . . . . . . . 106

3-9 The rc_def_proxy policy method contents . . . . . . . . . . . . . . . . . . . . . . 107

3-10 How to modify the rc_def_proxy policy method . . . . . . . . . . . . . . . . . . 107

3-11 Startup of Remote Control Proxy on AIX operating system. . . . . . . . . 109

3-12 The rcproxy.log: RC Target Proxy log file . . . . . . . . . . . . . . . . . . . . . . 110

3-13 The rcproxy.log: RC Controller Proxy log file . . . . . . . . . . . . . . . . . . . . 111

3-14 The netstat output collected on the RC Target Proxy . . . . . . . . . . . . . 111

3-15 The netstat output collected on the Controller Proxy . . . . . . . . . . . . . . 112

3-16 The output of the rcproxy -list command . . . . . . . . . . . . . . . . . . . . . . . 113

4-1 RC Controller Proxy configuration file . . . . . . . . . . . . . . . . . . . . . . . . . 129

4-2 RC Target Proxy configuration file example . . . . . . . . . . . . . . . . . . . . 130

4-3 Modification settings example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

4-4 The Relay.cfg after the installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

4-5 The Relay.cfg file after the changes . . . . . . . . . . . . . . . . . . . . . . . . . . 132

4-6 The rc_def_proxy policy method contents . . . . . . . . . . . . . . . . . . . . . . 132

4-7 The rcproxy.log: RC Target Proxy log file . . . . . . . . . . . . . . . . . . . . . . 133

4-8 The Relay.log: Relay log file contents . . . . . . . . . . . . . . . . . . . . . . . . . 134

4-9 The rcproxy.log: RC Controller Proxy log file . . . . . . . . . . . . . . . . . . . . 134

4-10 The netstat output collected on the RC Target Proxy . . . . . . . . . . . . . 135

4-11 The netstat output collected on the Relay . . . . . . . . . . . . . . . . . . . . . . 135

4-12 The netstat output collected on the Controller Proxy . . . . . . . . . . . . . . 136

4-13 The output of the rcproxy -list command . . . . . . . . . . . . . . . . . . . . . . . 137

5-1 The rcproxy.log file settings in the rcproxy.cfg file . . . . . . . . . . . . . . . . 155

5-2 The Target Proxy log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

5-3 The Controller Proxy log file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

5-4 The remcon.trc file for the Controller machine. . . . . . . . . . . . . . . . . . . 158

5-5 The Target Proxy log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

5-6 The Controller Proxy log file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

5-7 The remcon.trc file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

5-8 The rc_def_proxy policy method changes in order to fix the problem . 162

5-9 The Endpoint Proxy log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

5-10 The Relay log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

5-11 The Gateway Proxy log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

5-12 The Target Proxy log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

5-13 The Relay log file (instance used by remote control proxies) . . . . . . . 166

5-14 The Controller Proxy log file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

5-15 The remote control Target Proxy log file . . . . . . . . . . . . . . . . . . . . . . . 167

5-16 The remote control Relay log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

5-17 The remote control Controller log file. . . . . . . . . . . . . . . . . . . . . . . . . . 169

5-18 The Relay configuration file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

5-19 Wrong Target Proxy configuration file . . . . . . . . . . . . . . . . . . . . . . . . . 170

xii IBM Tivoli Remote Control Across Firewalls

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to:

IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law

THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.

: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.

COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces.

Trademarks

The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:

AIX® CT™

™

Illustra™ IBM® ibm.com®

The following terms are trademarks of other companies:

ActionMedia, LANDesk, MMX, Pentium and ProShare are trademarks of Intel Corporation in the United States, other countries, or both.

Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

C-bus is a trademark of Corollary, Inc. in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC.

Other company, product, and service names may be trademarks or service marks of others.

pSeries™ Redbooks™ Redbooks (logo) ™ SecureWay® SP™ SP1®

Tivoli Enterprise™ Tivoli Enterprise Console® Tivoli® TME 10™ TME®

xiv IBM Tivoli Remote Control Across Firewalls

Preface

System administrators and help desk personnel sometimes need access to remote PCs in order to resolve problems and assist users with important business applications. Most organizations will, at some point, need to expand their management of systems from their regular systems management environment to those that exist on the other side of firewalls. Tivoli® Remote Control allows a system administrator to control the keyboard and mouse input and monitor the display output of a remote machine, independently of the firewall architecture.

This book presents a concise documentation of known requirements for implementing the IBM Tivoli Remote Control 3.8 across firewalls.

This IBM® Redbook will prove invaluable for Tivoli system administrators and Tivoli designers and firewall administrators planning, designing, implementing, and operating a Remote Control environment that involves firewalls. The results from a variety of test scenarios are presented along with tabulated firewall configuration requirements for the various components involved in such solution.

The team that wrote this redbook

This redbook was produced by a team of specialists from around the world working at the International Technical Support Organization, Austin Center.

Edson Manoel is a Software Engineer at the IBM Corporation, International Technical Support Organization (ITSO), Austin Center, working as an IT Specialist in the Systems Management area. Prior to joining the ITSO, Edson worked in the IBM Software Group as a Tivoli Technology Ambassador and in IBM Brasil Professional Services Organization as a Certified IT Specialist. He was involved in numerous projects, designing and implementing systems management solutions for IBM customers and Business Partners. Edson holds a BSc degree in Applied Mathematics from Universidade de Sao Paulo, Brazil.

Francesca Balzi is a Tivoli Software Engineer at the IBM Tivoli Software Laboratory, in Italy. She has 7 years of experience in Customer Support. Her areas of expertise include problem determination and source identification in the client-server environment. She is currently performing Level 2 support for the EMEA Geo on IBM Tivoli Remote Control and IBM Tivoli Configuration Manager.

Sebastien Fardel is an Advisory IT Specialist at IBM Corporation, Global Services, Switzerland, acting as a Tivoli Architect in the Performance and Availability and Configurations and Operations areas. He has been in the IT industry since 1996 and has experience in IT infrastructure management, programming, and Systems Management area. His e-mail is sfa@ch.ibm.com.

Venkata Reddy is a software Engineer working for IBM Software Labs in Bangalore, India. He has three years of IT experience and is working as part of networking software group. He leads the firewall India team for providing Level3 support and Enhancements for IBM SecureWay® firewall. His areas of expertise include network security and firewalls.

Thanks to the following people for their contributions to this project:

Joanne Luedtke, Lupe Brown, Wade Wallace, and Chris Blatchley International Technical Support Organization, Austin Center

Yvonne Lyon International Technical Support Organization, San Jose Center

Silvia Giacone, Nicola Milanese, and Ugo Madama Remote Control Development and Verification Team, IBM Rome

Alan Hsu Market Manager - Remote Control, IBM Software Group Austin

Become a published author

Join us for a two- to six-week residency program! Help write an IBM Redbook dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You'll team with IBM technical professionals, Business Partners and/or customers.

Your efforts will help increase product acceptance and customer satisfaction. As a bonus, you'll develop a network of contacts in IBM development labs, and increase your productivity and marketability.

Find out more about the residency program, browse the residency index, and apply online at:

ibm.com/redbooks/residencies.html

xvi IBM Tivoli Remote Control Across Firewalls

Comments welcome

Your comments are important to us!

We want our Redbooks™ to be as helpful as possible. Send us your comments about this or other Redbooks in one of the following ways:

򐂰 Use the online Contact us review redbook form found at:

ibm.com/redbooks

򐂰 Send your comments in an Internet note to:

redbook@us.ibm.com

򐂰 Mail your comments to:

IBM Corporation, International Technical Support Organization Dept. JN9B Building 003 Internal Zip 2834 11400 Burnet Road Austin, Texas 78758-3493

Preface xvii

xviii IBM Tivoli Remote Control Across Firewalls

Part 1

Part 1 Concepts and

planning

2 IBM Tivoli Remote Control Across Firewalls

Chapter 1. Remote Control sessions

overview

System administrators often need to manage servers or workstations in distant secure locations — for example, in an outsourcing project. If a problem on one of these machines requires attention, the administrator traditionally has two options: try to resolve the problem over the telephone with an authorized person (with a high chance of miscommunication); or relocate to the user’s location for problem determination (which is often impractical). IBM Tivoli Remote Control allows an administrator to control the keyboard and mouse input and monitor the display output of a remote machine even in zones protected by any kind of network controlling process like firewalls. In addition, the administrator can just monitor, reboot the PC, or transfer files in a really simple way.

In this chapter we cover the following topics: 򐂰 Business perspectives for a solution using IBM Tivoli Remote Control across

firewalls

򐂰 An overview of the IBM Tivoli Remote Control functionality 򐂰 A detailed description of the IBM Tivoli Remote Control components that will

help to manage machines inside secure areas 򐂰 A detailed description of each type of communication used to exchange

information between the different zones

1.1 IBM Tivoli Remote Control overview

The purpose of this chapter is not only to explain how IBM Tivoli Remote Control works in general, but to emphasize its architecture and functionality in a firewall environment. Even though the architecture and implementation of IBM Tivoli Remote Control may differ when firewalls are involved from implementation to implementation, the IBM Tivoli Remote Control functionality will remain the same. Therefore, in order to fully understand how remote control sessions work across firewalls, it is important to understand how this works in a non-secure environment.

IBM Tivoli Remote Control (ITRC) provides a complete real-time solution for remote controlling the target systems. For all intents and purposes, the technician or administrator’s keyboard and mouse become the primary keyboard and mouse for the target system for the duration of a remote control session. Functionalities such as chat, reboot, and file transfer are available to the administrator.

IBM Tivoli Remote Control runs on top of the IBM Tivoli Management Framework. However, in the context of a firewalls environment, some other components must be installed in order to simplify and secure the way that communications are exchanged between the different components of IBM Tivoli Remote Control. Before continuing and defining the complete Remote Control process across firewalls, it is important to first know and understand the utility and functionality of each component of IBM Tivoli Remote Control and of IBM Tivoli Management Framework.

1.1.1 IBM Tivoli Management Framework components

The IBM Tivoli Management Framework enables you to install and create several management components (services) that allow you to manage the resources in your network. You can install any or all of these services, depending on your organizational needs. As a minimum, one TMR server must be installed. The following is a list of the management services provided by the Tivoli Management Framework and a brief description of each:

TMR Server The Tivoli Management Region (TMR) Server includes

the libraries, binaries, data files, and graphical user interface (GUI) needed to install and manage your Tivoli environment. The TMR Server maintains the Object DataBase and coordinates all communications with Tivoli managed systems, like Managed Nodes and Endpoints (through Tivoli Endpoint Gateways). The server also performs the authentication and verification necessary to ensure the security of Tivoli Enterprise™ data.

4 IBM Tivoli Remote Control Across Firewalls

Managed Node A Tivoli Managed Node runs the same software that runs

on a TMR Server. Managed Nodes maintain their own Object DataBases, which can be accessed by the TMR Server. When Managed Nodes communicate directly with other Managed Nodes, they perform the same communication or security operations as they would perform with the TMR Server. Although there is no clear distinction between managed systems and managing systems, the introduction of the Endpoints architecture leads to a paradigm shift. Managed Nodes are considered to be managing systems (hosting the desktop or running as a gateway), whereas endpoints are the managed systems.

Endpoint Manager The Endpoint Manager establishes and maintains the

relationship between an Endpoint and its assigned Gateway. It is involved in taking the Endpoint in charge when its assigned Gateway is no longer responding. It is also involved in identifying the Gateways that an Endpoint is assigned to when applications are trying to contact the Endpoint. The Endpoint Manager runs on top of the TMR Server and is automatically created during the TMR Server installation process.

Endpoint Gateway The Endpoint Gateway provides access to the Endpoint

methods and provides the communications with the TMR Server that the Endpoints occasionally require. A single Gateway can support communications with thousands of Endpoints and can launch methods on an Endpoint or run methods on the Endpoint’s behalf. A Gateway is created on an existing managed node.

Endpoint Proxy An Endpoint Proxy is an optional component that

emulates Endpoints to the Gateway to simplify the Tivoli communications in a firewall environment through a common port. The Endpoint Proxy funnels requests for specific Endpoints through a single TCP/IP port and passes it down to a Relay or a Gateway Proxy. This component is part of the Tivoli Firewall Security Toolbox and must be installed on the same network zone as the Tivoli Endpoint Gateway on which it is connected.

Chapter 1. Remote Control sessions overview 5

Relay The Relay component’s purpose is to pass information

sent to it up or down the chain to an Endpoint Proxy, Gateway Proxy, or other Relays. This component is optional and is part of the Tivoli Firewall Security Toolbox. It must be installed in the network zone between the Endpoint Proxy and the Gateway Proxy. Multiple Relays could be chained to allow this connection if Endpoint Proxy and Gateway Proxy are separated by multiple network zones. There can be multiple instances of the Relay running on the same machine.

Gateway proxy A Gateway Proxy is an optional component that

emulates a Gateway to the Endpoints to simplify the Tivoli communications in a firewall environment through a common port. The Endpoints are not explicitly aware of the fact that this destination is not truly a Gateway. This component is part of the Tivoli Firewall Security Toolbox and must be installed on the same network zone as the distant Endpoints.

Endpoint A Tivoli Management Agent (TMA) is any system that

runs an Endpoint service (or daemon). Typically, an Endpoint is installed on a machine that is not used for daily management operations. Endpoints run a very small amount of software and do not maintain a database. The majority of systems in most Tivoli Enterprise installations will be Endpoints.

Policy Region A Policy Region is a collection of Tivoli resources that are

governed by a common set of policies. A Policy Region is often created to represent a management domain or area of influence for one or more system administrators.

Administrator Tivoli Administrators are persons who will be responsible

for managing various aspects of enterprise wide systems management. Tivoli functionality allows administrative functions that may be performed at many levels and locations of the organization. Administrators may be individuals or groups of persons with different logins.

Collection The Collection is a container that groups objects on a

Tivoli Desktop, thus providing the Tivoli Administrator with as single view of related resources. Such Collections are defined when an Administrator has the need to centralize miscellaneous resources stored in different Policy Regions. A Collection provides a “shortcut” for using resources.

6 IBM Tivoli Remote Control Across Firewalls

For more information about TMR Server, Managed Node, Endpoint Gateway, Endpoint and Policy Region, refer to

Deployment Guide

For more information about Endpoint Proxy, Gateway Proxy and Relay, refer to Firewall Security Toolbox User ’s Guide, GC23-4826 and to Tivoli Enterprise Management Across Firewalls, SG24-5510.

, GC32-0803.

Tivoli Management Framework Planning for

1.1.2 IBM Tivoli Remote Control components

As already mentioned, the IBM Tivoli Remote Control is a client-server application that helps you take control over workstations on a network using a specific remote control technology. It could serve as a central location for monitoring and controlling machines at local or remote locations. The following is a definition list of the Remote Control components. Their installation is mandatory except for the Remote Control Proxies and the Remote Control Gateway, which are only used in environments where components of a Tivoli Management Region are separated by firewalls:

RC Server The Remote Control Server (RC Server) component is

installed on the TMR Server and on each Managed Node that will act as an Endpoint Gateway. It manages the Remote Control session request from a Remote Control Controller to a Remote Control Target until the connection between the two machines is successfully initiated.

RC Tool The Remote Control Tool (RC Tool) is the Remote

Control managed resource in the Tivoli Management Region and is associated with a Policy Region. This tool enables remote operations such as remote controlling or rebooting of a workstation, transferring files and chatting. Customizing the default Remote Control policies allows you to change the set of rules that will apply to the RC Tool within a Policy Region.

RC Policies The Remote Control Policies consist of a set of rules, the

so-called policy methods, that allows to change the default behavior and graphical appearance of Remote Control Tools.

RC Controller The Remote Control Controller component is

automatically installed on each Endpoint that initiates a Remote Control session. It will enable a Tivoli Administrator to take control of a remote target workstation to which it is linked over a network. This component is also known as Controller.

Chapter 1. Remote Control sessions overview 7

RC Target The Remote Control Target component is automatically

installed on each Endpoint when a session from a Remote Control Controller is initiated. This component is also known as Target.

RC Controller Proxy The Remote Control Controller Proxy is an optional

component which could be used to simplify the communication between Controllers and Targets in a firewall environment through a common port. In fact, this component simulates a Remote Control Controller to the Targets that are separated from the Controllers by firewalls. This component must be installed in the same network zone as the Targets. Nevertheless, this component could be either installed on top of a Endpoint/Gateway Proxy or as a Standalone component.

RC Target Proxy The Remote Control Target Proxy is an optional

component which could be used to simplify the communication between Controllers and Targets in a firewall environment through a common port. In fact, this component simulates Remote Control Targets to the Controllers that are separated from the Targets by firewalls. This component must be installed in the same network zone as Controllers. Nevertheless, this component could be either installed on top of a Endpoint/Gateway Proxy or as a Standalone component.

RC Gateway The Remote Control Gateway is an optional component

which could be used when direct link from the Controller to the Target is not authorized. Thus, in this case, a Remote Control Gateway needs to be installed on top of a Tivoli Endpoint Gateway.

1.1.3 Tivoli components and communication symbols

In the figures and scenarios that follow, we use the following set of symbols to denote the various components and type of communication for easy recognition:

Tivoli Management Region Server (blue line)

Endpoint Gateway, Remote Control Server, Endpoint Manager or Instance of the Tivoli Firewall Security Toolbox Relay

8 IBM Tivoli Remote Control Across Firewalls

Endpoint, Remote Control Controller or Remote Control Target

Policy Region (blue line)

Collection (blue line)

Remote Control Tool

Endpoint Proxy or Gateway Proxy (black line)

Remote Control Target Proxy or Remote Control Controller Proxy

Instance 1 of the Tivoli Firewall Security Toolbox Relay (black line)

Firewall

Network zone secured by a firewall (red line)

Tivoli Framework communication (black line)

Tivoli Remote Control session communication (blue line)

Tivoli proprietary protocol encapsulated in HTTP (green line)

Chapter 1. Remote Control sessions overview 9

1.1.4 Parent-Child concept

The hierarchy of the components of either the Tivoli Firewall Security Toolbox or the Remote Control Proxies (either RC Target Proxy or RC Controller Proxy) is presented in terms of a Parent-Child relationship. Such hierarchy is a subset of the whole Tivoli Top-Down hierarchy. It means that the starting point is the TMR Server and the ending point is the Endpoint. The components close to the TMR Server will be Parents and the ones close to the Endpoints will be Children. However, some components could, at the same time, be a Child and a Parent, as they are just in between the Parent-Child hierarchy. You must also notice that a Parent could have more than one Child but that a Child could only have one Parent.

As the Endpoint Proxy, which simulates Endpoints, is the closest element from the TMR Server, it is a Parent and, as it is directly connected to a Tivoli Gateway, it could not have any Parent. As the Gateway Proxy, which simulates a Tivoli Gateway, is the closest element from the Endpoints, it is a Child and as it the most closest component from the bottom of the hierarchy, it could not have any Child. A Relay could be either a Parent or a Child. When it is connected to an Endpoint Proxy or to another Relay, it becomes a Child of those components. In another way, when a Gateway Proxy or another Relay connects to a Relay, this one also becomes a Parent for these components.

In the case of Remote Control Proxies being installed on top of the Tivoli Firewall Security Toolbox components, the RC Proxy (either Target or Controller Proxy) installed on the Endpoint Proxy is a Parent of Relays or other RC Proxies. The RC Proxy installed on the Gateway Proxy is a Child of an RC Proxy installed on an Endpoint Proxy or a Relay. As no Remote Control components could be installed on the Relay, an RC Proxy could only be either a Parent or a Child, but not both at the same time.

If the Remote Control Proxies are installed as Standalone components, you have to decide on the Parent-Child role for each of the Proxies (Target and Controller Proxies). For configuration improvement, it is advised to configure the Target Proxy as the Parent and the Controller Proxy as the Child. This is because connection requests to the Target Proxy are done by the RC Controller. So, this request is always forwarded by the RC Target Proxy to the RC Controller Proxy. In this case, to logically respect a Top-Down hierarchy and to improve performance for the request, the RC Target should be the Parent.

Figure 1-1 depicts the Top-Down Proxy hierarchy when Remote Control Proxy components are installed on top of the Tivoli Firewall Security Toolbox.

10 IBM Tivoli Remote Control Across Firewalls

+ 198 hidden pages