VMware vCenter Configuration Manager - 5.4.1 Installation Manual

VMware vCenter Configuration Manager

Installation and Getting Started Guide

vCenter Configuration Manager 5.4.1

This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition. To check for more recent editions of this document, see

http://www.vmware.com/support/pubs.

EN-000740-00

vCenter Configuration Manager Installation and Getting Started Guide

You can find the most up-to-date technical documentation on the VMware Web site at:

http://www.vmware.com/support/

The VMware Web site also provides the latest product updates.

If you have comments about this documentation, submit your feedback to:

docfeedback@vmware.com

http://www.vmware.com/go/patents.

VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies.

VMware, Inc.

3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com

2 VMware, Inc.

Contents

About This Book 9

Preparing for Installation 11

Installation Manager 11 Installation Configurations 12 Tools Installation 12 General Prerequisites to Install VCM 12

Verify Hardware and Software Requirements 12 Verify Administration Rights 12 Set the Default Network Authority Account 12 Specify the Collector Services Account 13 Change the Collector Services Account Password in the Services Management Console 13 Change the Collector Services Account Password in the Component Services DCOM Config Console 13 Verify the VMware Application Services Account 14 Determine the VCM Remote Virtual Directory 14 Use Secure Communications Certificates 14 Understand Server Authentication 14 Verify the Foundation Checker System Checks 16 Install UNIX Patch for HP-UX 11.11 16

VCM Uses FIPS Cryptography 16

VCM Uses Microsoft Cryptographic Service Providers for Windows Machines 17 Cryptography for UNIX/Linux Platforms 17 Cryptography used in VCM Software Components 17 Supported Windows and UNIX Platforms 18

Installing VCM 19

Installing, Configuring, and Upgrading the OS Provisioning Server and Components 21

Restricted Network Environment 21 Install and Configure the OS Provisioning Server 21

Install the OS Provisioning Server 22 Set the vcmuser Password 24 Configure DHCP 25 Configure TFTP 26 Create a Windows Boot Image 26 Copy the VCM Certificate to the OS Provisioning Server for Linux Provisioning 27 Configure OS Provisioning Server Integration with the VCM Collector 28

Import Distributions into the OS Provisioning Server Repository 33

Create Directories for Windows Distributions 34 Import Windows Distributions 34 Import Linux/ESX Distributions 36

Using the basicimport Command Options 38 Working with Custom Linux ISO Distributions 38 Upgrade the OS Provisioning Server to 5.4.1 39

Before Upgrading the OS Provisioning Server 39

Upgrading the OS Provisioning Server 39

After Upgrading the OS Provisioning Server 39 Managing the OS Provisioning Server System Logs 40 ospctrl Command Options 40

VMware, Inc.

vCenter Configuration Manager Installation and Getting Started Guide

Upgrading or Migrating VCM 43

Upgrades 43 Migrations 43 Prerequisites to Migrate VCM 44 Back Up Your Databases 45 Back up Your Files 45 Export and Back up Your Certificates 45 Migrating VCM 46

Migrate Only Your Database 46

Replace Your Existing 32-Bit Environment with a Supported 64-bit Environment 47

Migrate a 32-bit Environment Running VCM 5.3 or Earlier to VCM 5.4.1 48

Migrate a 64-bit Environment Running VCM 5.3 or Earlier to VCM 5.4.1 49

Migrate a Split Installation of VCM 5.3 or Earlier to a Single-Server Installation 51

How to Recover Your Collector Machine if the Migration is not Successful 53 Upgrading VCM and Components 54

Upgrade VCM 55

Upgrade Existing Windows Agents 55

Upgrade Existing VCM Remote Clients 56

Upgrade Existing UNIX Agents 57

Upgrade VCM for Virtualization 60

Maintaining VCM After Installation 65

Customize VCM and Component-Specific Settings 65 Database Recovery Models 67 Configure Database File Growth 67 Configure Database Recovery Settings 68 Create a Maintenance Plan for SQL Server 2008 R2 69 Incorporate the VCM CMDB into your Backup and Disaster Recovery Plans 70

Getting Started with VCM Components and Tools 71

Understanding User Access 71

Running VCM as Administrator on the Collector 72 Log In to VCM 72 Getting Familiar with the Portal 73

General Information Bar 73

Portal Toolbar 74

Sliders 75

Getting Started with VCM 77

Discover, License, and Install Windows Machines 77

Verify Available Domains 78

Check the Network Authority 78

Assign Network Authority Accounts 79

Discover Windows Machines 79

License Windows Machines 80

Disable User Account Control for VCM Agent Installation 81

Install the VCM Windows Agent on Your Windows Machines 83

Enable UAC After VCM Agent Installation 89

Collect Windows Data 90

Windows Collection Results 91

Getting Started with Windows Custom Information 92 Discover, License, and Install UNIX/Linux Machines 111

Upgrade Requirements for UNIX/Linux Machines 112

Add UNIX/Linux Machines 112

License UNIX/Linux Machines 114

Install the Agent on UNIX/Linux Machines 114

VMware, Inc.

Contents

Collect UNIX/Linux Data 121

UNIX/Linux Collection Results 121 Discover, License, and Install Mac OS X Machines 122

Add Mac OS X Machines 123

License Mac OS X Machines 124

Install the Agent on Mac OS X Machines 124

Collect Mac OS X Data 129

Mac OS X Collection Results 131 Discover, Configure, and Collect Oracle Data from UNIX Machines 131

Discover Oracle Instances 132

Edit Oracle Instances 133

Collect Oracle Data 137

Oracle Collection Results 138 Customize VCM for your Environment 139 How to Set Up and Use VCM Auditing 139

Getting Started with VCM for Virtualization 141

Virtual Environments Configuration 141

ESX/ESXi Server Collections 142

vCenter Server Collections 143

vCloud Director vApp Virtual Machines Collections 143 Configure vCenter Server Data Collections 143

Configure vCenter Server Collection Prerequisites 143

Collect vCenter Server Data 145

vCenter Server Collection Results 146

Troubleshooting vCenter Server Data Collections 146 Configure Virtual Machine Host Collections 147

vCenter Server Collection Upgrade Considerations 147

Configure the Collector as an Agent Proxy 147

License and Configure Virtual Machine Hosts 148

Copy Files to the ESX/ESXi Servers 150

Collect Virtualization Data 151

Virtualization Collection Results 152 Configure vCloud Director vApp Virtual Machines Collections 152

Network Address Translation and vCloud Director vApp Discovery Rules 153

Generate vCloud Director Collection Credentials 155

Create vCloud Director Data Collection Filters 156

Collect vCloud Director Data 158

Discover vCloud Director vApp Virtual Machines 158

vCloud Director Collection Results 162 Configure the vSphere Client VCM Plug-In 163

Configuring the vSphere Client VCM Plug-In Integration Settings 164

Manage Machines from the vSphere Client 165

Troubleshooting the vSphere Client VCM Plug-In Registration 165

Getting Started with VCM Remote 167

VCM Remote Management Workflow 167

Configuring VCM Remote Connection Types 167

Using Certificates With VCM Remote 168 Configure and Install the VCM Remote Client 168

Configure the VCM Remote Settings 168

Install the VCMRemote Client 171

Connect VCM Remote Client Machines to the Network 178

VCM Remote Collection Results 179

Getting Started with VCM Patching 181

VMware, Inc.

vCenter Configuration Manager Installation and Getting Started Guide

VCM Patching for Windows and UNIX/Linux Machines 181

VCM Patching for Windows Machines 181

VCM Patching for UNIX and Linux Machines 182

Minimum System Requirements 182

UNIXand Linux Patch Assessment and Deployment 182

Getting Started with VCM Patching 184

vCenter Software Content Repository Tool 190

Running VCM Patching Reports 197

Customize Your Environment for VCMPatching 198

Getting Started with Operating System Provisioning 199

OS Provisioning Components 199

How OS Provisioning Works 200 Provision Target Machines with Operating System Distributions 201

Collect OS Distributions 201

Discover Provisionable Machines 202

Provision Machines with Operating System Distributions 202 Provisioned Machines Results 213 Re-Provision Machines 214

Getting Started with Software Provisioning 217

Using Package Studio to Create Software Packages and Publish to Repositories 217

Software Repository for Windows 217

Package Manager for Windows 217

Software Provisioning Component Relationships 218 Install the Software Provisioning Components 218

Install Software Repository for Windows 219

Install Package Studio 220

Install Package Manager on Managed Machines 222 Using Package Studio to Create Software Packages and Publish to Repositories 223

Creating Packages 223 Using VCM Software Provisioning for Windows 225

Collect Package Manager Information from Machines 226

Collect Software Repository Data 226

Add Repository Sources to Package Managers 227

Install Packages 228 Related Software Provisioning Actions 230

Viewing Provisioning Jobs in the Job Manager 230

Create Compliance Rules Based on Software Provisioning Data 231

Create Compliance Rules Containing Software Provisioning Remediation Actions 232

Getting Started with VCM Management Extensions for Assets 235

Configure Asset Data Fields 235

Review Available Asset Data Fields 236

Add an Asset Data Field 236

Edit an Asset Data Field 237

Delete a VCMMXA Data Field 238

Change the Order of Asset Data Columns 238

Refresh Dynamic Asset Data Fields 239 Configure Asset Data Values for VCM Machines 239 Configure Asset Data for Other Hardware Devices 240

Add Other Hardware Devices 240

Add Multiple Similar Other Hardware Devices 241

Edit Asset Data for Other Hardware Devices 241

Edit Asset Data Values for Other Hardware Devices 242

Delete Other Hardware Devices 242 Configure Asset Data for Software 243

VMware, Inc.

Contents

Add Software Assets 243

Add Multiple Similar Software Assets 244

Edit Asset Data for Software 245

Edit Asset Data Values for Software 245

Delete Software Data 246

Getting Started with VCM Service Desk Integration 247

Configure Service Desk Integration 247 View Service Desk Integration in the Console 247 View Service Desk Integration in Job Manager 248

Getting Started with VCM for Active Directory 249

Configure Domain Controllers 249

Verify Available Domains 250

Check the Network Authority Account 250

Assign Network Authority Accounts 251

Discover Domain Controllers 251

License Domain Controllers 252

Install the VCM Windows Agent on Your Domain Controllers 253

Collect Domain Controller Data 254 Configure VCM for Active Directory as an Additional Product 255

Install VCM for Active Directory on the Domain Controllers 255

Run the Determine Forest Action 256

Run the Domain Controller Setup Action 256 Collect Active Directory Data 257 Active Directory Collection Results 258

Installing and Getting Started with VCM Tools 261

Install the VCM Tools Only 261 VCM Import/Export and Content Wizard Tools 262

Run the Import/Export Tool 263

Run the Content Wizard to Access Additional Compliance Content 263 Run the Deployment Utility 263 Package Studio 264 Foundation Checker 264

Index 265

VMware, Inc.

vCenter Configuration Manager Installation and Getting Started Guide

VMware, Inc.

About This Book

The VMware vCenter Configuration Manager Installation and Getting Started Guide describes the steps necessary for a successful VCM installation.

This document contains the following information:

Preparing for the VCM installation

Installing VCM

Maintaining VCM after installation

Getting started with VCM and its components

Read this document and complete the associated procedures to prepare for a successful installation.

The VMware vCenter Configuration Manager Installation and Getting Started Guide applies to VCM, Foundation Checker, and Service Desk Connector.

Intended Audience

This information is written for experienced Windows or UNIX/Linux/Mac OS X system administrators who are familiar with managing network users and resources and with performing system maintenance.

To use this information effectively, you must have a basic understanding of how to configure network resources, install software, and administer operating systems. You also need to fully understand your network’s topology and resource naming conventions.

Document Feedback

VMware welcomes your suggestions for improving our documentation. If you have comments, send your feedback to docfeedback@vmware.com.

VMware VCM Documentation

The vCenter Configuration Manager (VCM) documentation consists of the VCM Hardware and Software Requirements Guide, VCM Foundation Checker User's Guide, VCM Installation and Getting Started Guide, VCM Troubleshooting Guide, VCM online Help, and other associated documentation.

VMware, Inc. 9

vCenter Configuration Manager Installation and Getting Started Guide

Technical Support and Education Resources

The following technical support resources are available to you. To access the current version of this book and other books, go to http://www.vmware.com/support/pubs.

Online and Telephone Support

Support Offerings To find out how VMware support offerings can help meet your business needs,

VMware Professional Services

To use online support to submit technical support requests, view your product and contract information, and register your products, go to

http://www.vmware.com/support.

Customers with appropriate support contracts should use telephone support for priority 1 issues. Go to http://www.vmware.com/support/phone_support.html.

go to http://www.vmware.com/support/services.

VMware Education Services courses offer extensive hands-on labs, case study examples, and course materials designed to be used as on-the-job reference tools. Courses are available onsite, in the classroom, and live online. For onsite pilot programs and implementation best practices, VMware Consulting Services provides offerings to help you assess, plan, build, and manage your virtual environment. To access information about education classes, certification programs, and consulting services, go to http://www.vmware.com/services.

10 VMware, Inc.

Preparing for Installation

You must prepare your environment before you install VCM components and tools.

Prerequisit es

Verify that your environment meets the security requirements. See the VCM Security Environment Requirements White Paper on the Download VMware vCenter Configuration Manager Web site.

Verify that your hardware and software configuration meets the requirements to install VCM. See the VCM Hardware and Software Requirements Guide.

Verify that your hardware and software meet the requirements to install VCM and install and run the standalone VCM Foundation Checker. See "Installing and Getting Started with VCM Tools" on page

261.

To prepare your environment, familiarize yourself with the following topics.

Installation Manager: Installs and activates VCM components and tools.

Installation Configurations: Describes supported installation configurations.

Tools Installation: Lists the installed VCM tools.

General Prerequisites to install VCM: Describes prerequisites that you must perform before you install

VCM.

Installation Manager

The VCM Installation Manager installs new versions of VCM components and tools and upgrades existing versions. Installation Manager performs several actions.

Checks managed machines to ensure that they meet the hardware and software prerequisites for the installation.

Confirms the license file that you apply during the installation.

Installs the components and tools in the appropriate order on your machines.

Tests each installation step to verify that all components install successfully and that licensed components activate successfully.

Installation Manager operates with minimal user input and reports on progress during the installation process. All VCM components are installed. Only components that you purchased are licensed. You can purchase more licenses later to activate the additional installed components.

If you are upgrading, see "Upgrading or Migrating VCM" on page 43.

VMware, Inc. 11

vCenter Configuration Manager Installation and Getting Started Guide

Installation Configurations

Understand the installation configurations, configure your hardware, and install the prerequisite software. See the VCM Hardware and Software Requirements Guide.

Split installations are not supported. To migrate a split installation of VCM 5.3 or earlier to a single-server installation, see "Upgrading or Migrating VCM" on page 43. For more information, contact VMware Technical Support.

Tools Installation

The VCM Installation Manager installs several tools.

Foundation Checker

Import/Export Tool and Content Wizard Tool

Package Studio

You may install VCM tools separately on a non-Collector machine. See "Installing and Getting Started with

VCM Tools" on page 261.

General Prerequisites to Install VCM

Perform the general prerequisites to ensure that your environment is adequately prepared before you use Installation Manager to install VCM.

Verify Hardware and Software Requirements

Your hardware and software configuration must meet the requirements in the VCM Hardware and Software Requirements Guide.

Verify Administration Rights

Verify that the user account of the person who performs the installation or upgrade has all of the following rights.

System administrator on the machines on which the installation or upgrade is performed, and

System administrator on the database instance to be used, and

Member of a domain.

The installing user account must not be the account used to run SQL Server services. In addition, after installation, do not create a VCM user that uses the SQL Server services account credentials.

Set the Default Network Authority Account

Define the network authority account in the Local Administrators group on each Collector machine before you install VCM. See the VCM Hardware and Software Requirements Guide.

You specify the default network authority account during VCM installation. The default network authority account can be a system administrator account, such as a Domain Admin in the Local Admin Group.

The Local System account, NT AUTHORITY\System, has unrestricted access to all local system resources. This account is a member of the Windows Administrators group on the local machine and a member of the SQL Server sysadmin fixed server role.

12 VMware, Inc.

Preparing for Installation

If the NT AUTHORITY\System account does not have access to the VCM installation binary files, the installation results in an “access denied” error. You must grant access to the NT AUTHORITY\System account from the installation source directory and then run the installation again. Right-click the folder, select the Security tab, and verify that the user or user’s group has Full Control of the file/folder.

To change the network authority account later in VCM, click Administration and select Settings > Network Authority.

Specify the Collector Services Account

You specify the Collector Services Account during VCM installation. The account can be a system administrator account and must exist in the Local Administrators group on the Collector machine. The account must not be the Local System account.

If the password for the account changes, you must change the password in the Services Management console and the Component Services DCOM Config console.

Change the Collector Services Account Password in the Services Management Console

If the password for your Collector services account changes, you must change the services password in the Services Management Console.

Procedure

1. Click Start.

2. Select All Programs > Administrative Tools >Services.

3. Locate all of the services that use the collector services account to log on.

4. Right-click each of these services and select Properties.

5. Click the Log On tab and update the password field to reflect your new password.

6. Click OK.

Change the Collector Services Account Password in the Component Services DCOM Config Console

If the password for your Collector services account changes, you must change the services password in the Component Services DCOM Config console.

Procedure

1. Click Start.

2. Select All Programs > Administrative Tools >Component Services.

3. Expand Component Services and Computers.

4. Expand My Computer and select DCOM Config.

5. Right click LicenseDcom and select Properties.

6. Click the Identity tab and update the password field to reflect your new password.

7. Click OK.

VMware, Inc. 13

vCenter Configuration Manager Installation and Getting Started Guide

Verify the VMware Application Services Account

Verify that the VMware Application Services Account is a domain user. This account has full administrative authority for the CSI_Domain database.

IMPORTANT Never use this account as a VCM login or for any other purpose.

Determine the VCM Remote Virtual Directory

You specify the VCM Remote Virtual Directory during VCM installation. You can change the account later using the IIS Management console.

IMPORTANT When you specify the VCM Remote Virtual Directory, to minimize security risks to your

accounts, always use an account that differs from the account used for your Default Network Authority Account or your Services Account.

Use Secure Communications Certificates

VCM uses Transport Layer Security (TLS) to secure all HTTP communication with all Windows Agents and UNIX Agents in HTTP mode. TLS uses certificates to authenticate the Collector and Agents to each other. During VCM installation, you must specify the Collector and Enterprise certificates. If you use your own certificates, you must familiarize yourself with the certificate names in advance so that you can select them during installation.

A valid Collector certificate must be:

Located in the local machine personal certificate store.

Valid for Server Authentication. If any Enhanced Key Usage extension or property is present, it must include the Server Authentication OID 1.3.6.1.5.5.7.3.1. If the Key Usage extension is present, it must include DIGITAL_SIGNATURE.

Active, and not expired.

If you do not want to use your own certificates, you can have Installation Manager generate the Collector and Enterprise certificates for you, select the Generate option during the installation.

If you install more than one Collector that will communicate with the same Agent(s), or if you plan to replace or renew your certificates later, you must follow the special considerations to generate and select certificates in VCM Installation Manager. See the Transport Layer Security Implementation for VCM white paper on the Download VMware vCenter Configuration Manager Web site.

Understand Server Authentication

VCM supports Server Authentication, which is a method to authenticate the server to the client. In VCM environments where TLS is used, VCM Agents verify the identity of the Collectors by using and verifying certificates over HTTP.

The server typically authenticates a client or user by requiring information such as a user name and password. When Server Authentication is used, the client or user verifies that the server is valid. To accomplish this verification, the server provides a certificate issued by a trusted authority, such as Verisign. If your client Web browser has the Verisign Certified Authority certificate in its trusted store, the Web browser can trust that the server is actually the Web site you access.

14 VMware, Inc.

Preparing for Installation

To guarantee the identity of servers and clients, TLS uses certificates that are managed by a public key infrastructure (PKI). A certificate is a package that contains a public key, information that identifies the owner and source of that key, and one or more certifications (signatures) to verify that the package is authentic. To sign a certificate, an issuer adds information about itself to the information that is already contained in the certificate request. The public key and identifying information are hashed and signed using the private key of the issuer’s certificate.

Certificates are defined by the X.509 RFC standard, which includes fields that form a contract between the creator and consumer. The Enhanced Key Usage extension specifies the use for which the certificate is valid, including Server Authentication.

Enterprise and Collector Certificates

An Enterprise Certificate and one or more Collector Certificates enable secure HTTP Collector and Agent communication in VCM. The Enterprise Certificate enables VCM to operate in a multi-Collector environment. Agents have the Enterprise Certificate in their trusted certificate stores, and they use the Enterprise Certificate to validate any certificate issued by the Enterprise Certificate. All Collector Certificates are expected to be issued by the Enterprise Certificate, which is critical in environments where a single Agent is shared between two Collectors.

Server authentication is required to establish a TLS connection with an Agent. All VCM Collectors should have a common Enterprise Certificate. Each Collector Certificate is issued by the Enterprise Certificate, and is capable of Server Authentication. Collector Certificates in VCM must adhere to the requirements for secure communications certificates. See "General Prerequisites to Install VCM" on page 12.

The Collector Certificate initiates and secures a TLS communication channel with an HTTP Agent. The Agent must be able to establish that the Collector Certificate can be trusted, which means that the Collector Certificate is valid and the certification path starting with the Collector Certificate ends with a trusted certificate. By design, the Enterprise Certificate is installed in the Agent’s trusted store. The trust chain ends with the Enterprise Certificate.

A Collector Certificate can issue Agent certificates. When all Collector Certificates are issued by the same Enterprise Certificate, any Agent Certificate may be issued by any Collector Certificate, and all Agents can trust all Collectors. All Collectors can validate all Agent Certificates. Agent Certificates are used for Mutual Authentication only. VCM supports Mutual Authentication, which requires interaction with VMware Technical Support and a Collector Certificate that has certificate signing capability.

The Collector Certificate and associated private key must be available to the Collector. This certificate is stored in the local machine personal system store.

Delivering Initial Certificates to Agents

VCM Agents use the Enterprise Certificate to validate Collector Certificates. The Agent must have access to the Enterprise Certificate as a trusted certificate. In most cases, VCM delivers and installs the Enterprise Certificate as needed.

Installing the Agent from a Disk (Windows only)

The VCM Installation DVD does not contain customer-specific certificates. If HTTP is specified, the manual VCM installer requests the location of the Enterprise Certificate file during the installation. You must have the Enterprise Certificate file available at installation time. You can copy the certificate file, which has a .pem extension, from the CollectorData folder on the Collector. You must copy the certificate file when you run the manual installer directly using CMAgentInstall.exe or when you use the Agent Only option in the DVD auto-run program.

Using CMAgentInstall.exe to Install the Agent (Windows only)

VMware, Inc. 15

vCenter Configuration Manager Installation and Getting Started Guide

The CMAgentInstall.exe or CMAgent[version].msi is the manual Agent installer program. The manual installer requests the location of the Enterprise Certificate file when HTTP is specified. You must have the Enterprise Certificate file available at installation time. You can copy the certificate file from the CollectorData folder on the Collector.

Using the MSI Install Package

When you specify HTTP, the MSI Agent install package also requires access to the .pem file.

Installing the Agent for UNIX/Linux

See "Install the Agent on UNIX/Linux Machines" on page 114.

Installing the Agent Using a Provisioning System

For Windows, the manual installation program is available in EXE and MSI formats. Both versions allow you to specify the Enterprise Certificate file by using a command line switch. You may omit the certificate installation step by using a command line switch.

When these programs are run through a provisioning system, you must ensure that the Enterprise Certificate is available and secure, and configure the program options appropriately. Alternatively, you may choose to send the Enterprise Certificate to Agents by some other means and configure the provisioning system to omit certificate installation.

For UNIX/Linux, each UNIX/Linux installation package is targeted for one or more supported platforms. To install the UNIX/Linux Agent using a provisioning system, extract the installation package and then deploy the extracted file with the provisioning system. The Enterprise Certificate is embedded in the installation package on the Collector.

For more information about installing the Agent on UNIX/Linux machines, and UNIX/Linux packages and platforms, see "Install the Agent on UNIX/Linux Machines" on page 114.

Verify the Foundation Checker System Checks

Installation Manager runs Foundation Checker automatically during the VCM installation. Foundation Checker checks your Collector to verify that all of the prerequisites are satisfied for a successful installation.

When Foundation Checker runs as part of the Installation Manager process, it verifies component-specific issues against VCM. Foundation Checker captures common issues that are difficult to remediate and identifies issues with the components and version of VCM being installed. Foundation Checker must run without generating errors before you install VCM. For more information about the standalone Foundation Checker, see "Installing and Getting Started with VCM Tools" on page 261) and the VCM Foundation Checker User's Guide on the Download VMware vCenter Configuration Manager Web site.

Install UNIX Patch for HP-UX 11.11

If you install the VCM Agent on HP-UX 11.11 platforms, install patch PHSS_30966. For assistance, contact VMware Technical Support.

VCM Uses FIPS Cryptography

VCM incorporates cryptographic service providers that conform to Federal Information Processing Standards (FIPS) standards. The FIPS standards are developed by the US National Institute of Standards (NIST) and the Canadian Communications Security Establishment (CSE).

VCM supports the following FIPS standards.

16 VMware, Inc.

Preparing for Installation

FIPS 140-2: Security Requirements for Cryptographic Modules

FIPS 46-3: Data Encryption Standard (DES)

FIPS 81: DES Modes of Operation

FIPS 113: Computer Data Authentication

FIPS 171: Key Management

FIPS 180-1: Secure Hash Standard (SHA-1)

FIPS 186-2: Digital Signature Standard (DSA) and Random Number Generation (RNG)

FIPS 198: Message Authentication Codes (MACs) using SHA-1

FIPS 197: Advanced Encryption Standard (AES) Cipher

FIPS 200: Federal Information Security Management Act (FISMA)

SP 800-2: Public Key Cryptography (including RSA)

SP 800-20: Triple DES Encryption (3DES) Cipher

VCM Uses Microsoft Cryptographic Service Providers for Windows Machines

On Windows machines, VCM uses cryptography using the Microsoft CryptoAPI, which is a framework that dispatches to Microsoft Cryptographic Service Providers (CSPs). CSPs are not shipped with VCM or installed by VCM, but instead are part of the security environment that is included with Microsoft Windows. In the configurations supported by VCM, these CSPs are FIPS 140-2 validated.

For a current table of FIPS certificate numbers, see the FIPS 140 Evaluation in the online Microsoft Library.

Cryptography for UNIX/Linux Platforms

On UNIX/Linux platforms, the VCM Agent uses the cryptography of the OpenSSL v0.9.7 module. This cryptographic library is installed with the VCM Agent.

Cryptography used in VCM Software Components

VCM uses software components that also use cryptography.

Microsoft IIS, Internet Explorer, and SChannel (SSL/TLS) systems call the CryptoAPI, and therefore use the Windows FIPS-validated modules.

VCM for Virtualization uses ActiveX COM components from WeOnlyDo! Software (WOD) for SSH and SFTP services.

WOD uses the FIPS 140-2 compliant OpenSSL library.

Table 1–1. Installed or Used Crytography Modules

System

Platform

Open SSLFIPS 1.1.2

Open SSLFIPS 1.1.1

Open SSLCrypt 0.9.7

Crypto++ Crypto

API

UI Windows Used

VCMServer Windows Installed Used

Virt Proxy Windows Installed Used

AD Agent Windows Used

Win Agent Windows Used

VMware, Inc. 17

vCenter Configuration Manager Installation and Getting Started Guide

System

UNIX Agent HP/UX Installed Installed

ESX Server All No cryptography modules are used or installed on ESX.

Platform

AIX Installed Installed

Solaris Installed Installed

Debian Installed Installed

Red Hat Installed Installed

SUSE Installed Installed

Open SSLFIPS 1.1.2

Open SSLFIPS 1.1.1

Open SSLCrypt 0.9.7

Crypto++ Crypto

API

Supported Windows and UNIX Platforms

For a list of supported Windows and UNIX platforms and architectures, see the VCM Hardware and Software Requirements Guide. For information about TLS, see the Transport Layer Security (TLS) Implementation for VCM white paper on the Download VMware vCenter Configuration Manager Web site.

18 VMware, Inc.

Installing VCM

Use Installation Manager to install VCM and all of its components and tools. To install only the VCM tools, see "Installing and Getting Started with VCM Tools" on page 261.

The VMware vCenter Configuration Manager (VCM) Installation Manager is a standalone application that checks your machine to confirm that it is properly configured, installs VCM, and configures licensed components during the installation process.

VCM 5.4.1 supports 64-bit environments that include 64-bit hardware, the 64-bit Windows Server 2008 R2 operating system, and SQL Server 2008 R2.

When you install VCM and related components, the default settings might not fit your configuration exactly. You must read the information that appears for each configurable component and supply the appropriate information. If you migrate VCM or SQL Server, or migrate to a 64-bit system, see

"Upgrading or Migrating VCM" on page 43.

CAUTION The installation process adds the %windir%\Installer\ folder, which contains VCM related MSI files. Do not move or delete the content of this folder. If you delete the content, you will not be able to use Installation Manager to upgrade, repair, or uninstall VCM.

Prerequisit es

Review the list of supported platforms in the VCM Hardware and Software Requirements Guide.

Before you migrate VCM to VCM 5.4.1, read Migrating VCM and Related Components.

VMware, Inc. 19

vCenter Configuration Manager Installation and Getting Started Guide

Procedu re

1. To install VCM, insert the installation disk into the Windows machine.

The initial installation screen appears and displays several options. If the installation screen does not appear automatically, or if you began the installation from a network location, navigate to the disk root directory or the file share and double-click setup.exe.

2. Select an installation option.

Option Description

Run Installation Manager Starts Installation Manager and begins the installation.

View Help Displays the Installation Manager Help, which describes the selections that appear

during the installation.

Browse Contents of

Installation CD

Contact Support Team Displays instructions to contact VMware Technical Support.

Exit Closes Installation Manager.

Starts Windows Explorer and displays the content of the installation disk, which

includes documentation.

3. Follow the steps through the wizard to complete the installation.

For details about the installation options, open the Installation Manager online help.

What to do next

When the installation is finished, configure SQL Server database file growth and database recovery settings to tune your VCM database. See "Maintaining VCM After Installation" on page 65.

20 VMware, Inc.

Installing, Configuring, and Upgrading the OS Provisioning Server and Components

The Operating System (OS) Provisioning Server serves as a repository of imported OS distributions and manages the installation of the distributions on target machines. The installation of the distributions is part of the OS provisioning function in VCM, which identifies machines that can be provisioned and initiates the OS provisioning on the target machines.

You install and configure the OS Provisioning Server on a Red Hat server. After configuring the server, you import the operating system ISO files. The database manages the metadata about the OS distributions and the ISO files are saved in the OS Provisioning Server repository. After you import the distributions, the server performs the installation process, which is managed in VCM. See "Getting Started with

Operating System Provisioning" on page 199 for provisioning machines instructions.

You cannot directly upgrade from OS Provisioning Server 5.4 to 5.4.1. Nor is OS Provisioning Server 5.4 compatible with VCM 5.4.1. You must install the new 5.4.1 OS Provisioning Server components, configure the server, and import the operating system ISO files into the new database structure. See "Upgrade the

OS Provisioning Server to 5.4.1" on page 39.

When the OS Provisioning Server is installed and configured, consult the VCM Backup and Disaster Recovery Guide and create a backup plan for your server and files.

Troubleshooting information is available in the VCM Troubleshooting Guide.

Restricted Network Environment

To maintain security during the OS provisioning process, install and run your OS Provisioning Server in a private or restricted network. When you provision target machines, you connect the machines to this private network. See VCM Security Environment Requirements.

Install and Configure the OS Provisioning Server

You install the OS Provisioning Server and configure the components used to manage your operating system distributions. After you configure the components, you import the distributions and use VCM to install them on target machines.

Procedure

VMware, Inc. 21

vCenter Configuration Manager Installation and Getting Started Guide

1. "Install the OS Provisioning Server" on page 22

Using the supplied media or media images, install the OS Provisioning Server and run the command to create the distribution repository.

2. "Set the vcmuser Password" on page 24

Configure the vcmuser to use when you import distributions into the OS Provisioning Server repository and for communication between VCM and the OS Provisioning Server.

3. "Configure DHCP" on page 25

When you configure a private, isolated network that is used specifically for provisioning, the OS Provisioning Server uses the DHCP server it installed to provide addresses and network boot information to nodes connected to the network.

4. "Configure TFTP " on page 26

The OS Provisioning Server provides TFTP services that run on the provisioning network. You must configure the TFTP server to listen on the private OS provisioning network interface.

5. "Create a Windows Boot Image" on page 26

Create a Windows boot image and copy it to the OS Provisioning Server. You create the image on a Windows 2008 or Windows 7 machine, and copy the files to the OS Provisioning Server.

6. "Copy the VCMCertificate to the OS Provisioning Server for Linux Provisioning" on page 27

If you use the OS Provisioning Server to install Linux distributions, you must copy the VCM certificate file to the OS Provisioning Server to ensure the certificate is included with the Agent when OS Provisioning Server creates the configured session prior to provisioning.

7. "Configure OS Provisioning Server Integration with the VCM Collector" on page 28

The integration between VCM and the OS Provisioning Server uses Stunnel to establish secure communication between and the SOAP services of the two components.

Install the OS Provisioning Server

Using the supplied media or media images, install the OS Provisioning Server and run the command to create the distribution repository.

VCM OS provisioning supports a single instance of VCM with a single instance of the OS Provisioning Server.

Prerequisites

Install VCM. See "Installing VCM" on page 19.

Ensure the target machine meets the prerequisites specified in the VCM Hardware and Software Requirements Guide.

Determine whether you are installing the OS Provisioning Server as an attended or unattended installation. To run an unattended installation, use the ./autoinstall -a y command. This procedure is based on an attended installation.

22 VMware, Inc.

Installing, Configuring, and Upgrading the OS Provisioning Server and Components

Procedure

1. On the target machine, log in as root.

2. Mount the VCM-OS-Provisioning-Server-<version number>.iso by attaching or mounting the image.

When you mount the image, do not use the no exec option.

3. Type cd /<path to mounted OS Provisioning Server.iso> to change the directory to the location of the image.

4. Run the ./INSTALL-ME command to install server.

5. In the Nixstaller window, click Next.

6. In the dialog box, click Continue.

7. In the dialog box, click Close when the installation finishes.

8. In the Nixstaller window, click Finish.

9. Run the service FastScale status command to verify that the installation completed successfully.

A successful installation displays the following results. PID values vary.

rsyslogd (pid 3335) is running...

fsmesgd (pid 3517) is running...

fsrepod (pid 3683) is running...

fsadmin (pid 12618) is running...

dhcpd is stopped

tftpd (pid 12057) is running

fsjobd (pid 4237) is running...

fshinvd (pid 4249) is stopped...

stunnel (pid 4262 4261 4260 4259 4258 4257) is running...

An unsuccessful installation displays FastScale: unrecognized service or several of the above mentioned services are not running. Review the logs to determine possible problems.

10. Run the /opt/FastScale/sbin/create-repository command.

This action updates the repository database and destroys any existing repository information

11. Reboot the OS Provisioning Server to ensure that all related services are started in the correct order.

12. Run the service FastScale status command to verify the OS Provisioning Server services after reboot.

A successful installation displays the services and their PIDs as running.

What to do next

To ensure proper security, you must set the password for the vcmuser. See "Set the vcmuser Password"

on page 24.

(Optional) Add the OS Provisioning Server maintenance commands to the root user's path. The OS Provisioning Server modifies the default shell profiles by adding /opt/FastScale/sbin to the root account. When the user is root, the maintenance commands in /opt/FastScale/sbin are available in the default path and are available when the profile is reloaded.

VMware, Inc. 23

vCenter Configuration Manager Installation and Getting Started Guide

Uninstall the OS Provisioning Server

Uninstalling the OS Provisioning Server removes the provisioning application from the machine on which it is installed. You must mount the OS Provisioning Server media and run the uninstall command.

CAUTION The uninstall process removes the application and deletes all the data in the database.

Procedu re

1. On the OS Provisioning Server, log in as root.

2. Mount the OS Provisioning Server ISO by attaching or mounting the image.

3. Type cd /<path to OS Provisioning Server.iso> to change the directory to the location of the image.

4. Run the ./UNINSTALL-ME command to uninstall the application.

5. Type yes.

The uninstall process completes and generates a log. See the example log.

[Thu Jul 22 08:57:06 IST 2010] UNINSTALL-ME: Starting uninstallation of VCM OS

Provisioning Server...

[Thu Jul 22 08:57:08 IST 2010] UNINSTALL-ME: FastScale service is running

[Thu Jul 22 08:57:08 IST 2010] UNINSTALL-ME: Stopping FastScale service

[Thu Jul 22 08:57:08 IST 2010] UNINSTALL-ME: Command : /sbin/service FastScale

stop

Shutting down FSnetfs: [ OK ]

Shutting down FSsyslog: [ OK ]

Shutting down FSmesgd: [ OK ]

Shutting down FSdhcpd: [ OK ]

..........

[Thu Jul 22 09:00:44 IST 2010] UNINSTALL-ME: Uninstallation complete!

Set the vcmuser Password

Configure the vcmuser to use when you import distributions into the OS Provisioning Server repository and for communication between VCM and the OS Provisioning Server.

Do not delete the user or change the permissions, but you must set the vcmuser password based on your corporate standards.

Prerequisites

Verify that the OS Provisioning Server is installed. See "Install the OS Provisioning Server" on page 22.

Procedure

1. On the OS Provisioning Server, log in as root.

2. Run the passwd vcmuser command.

3. Type and confirm the new password.

What to do next

Configure DHCP with your local settings. See "Configure DHCP" on page 25.

24 VMware, Inc.

Installing, Configuring, and Upgrading the OS Provisioning Server and Components

Configure DHCP

Prerequisites

Determine whether you are using a private network (recommended) or shared network (supported, but not recommended). If you are provisioning systems on a shared network, you probably have a DHCP server on the network. Disable the OS Provisioning Server's DHCP server and configure your regular DHCP server to provide network boot information for machines to be provisioned. See "Configure a

DHCP Server Other Than the OS Provisioning Server" on page 25 .

Procedure

1. Open /opt/FastScale/etc/dhcpd.conf.

2. Configure the settings for your environment.

Option Description

subnet

The IP address subnet of the private network interface.

Default value is 10.11.12.0.

netmask

The netmask of the subnet.

Default value is 255.255.255.0.

range

The range of allocated IP addresses for the provisioned nodes.

Default value is 10.11.12.100–10.11.12.200.

broadcast-address

The broadcast address on the subnet.

Default value is 10.11.12.255.

next-server

The IP address of the private network interface.

Default value is 10.11.12.1.

What to do next

Configure the TFTP server to work with the provisioning environment. See "Configure TFTP " on page 26.

Configure a DHCP Server Other Than the OS Provisioning Server

To configure your system to work with a DHCP server other than the one on the OS Provisioning Server, you turn off the OS Provisioning Server DHCP server and configure your corporate DHCP server to connect to the OS Provisioning Server after nodes connect and NetBoot (PXE) starts. The nodes download the boot kernel from the OS Provisioning Server through TFTP.

Procedu re

1. On the OS Provisioning Server, log in as root.

2. Open /etc/sysconfig/FSdhcpd.

3. Change DHCPD_CONF=/opt/FastScale/etc/dhcpd.conf to DHCPD_

CONF=/opt/FastScale/etc/dhcpd.conf.none

This change prevents the DHCP from resetting after a reboot.

VMware, Inc. 25

vCenter Configuration Manager Installation and Getting Started Guide

4. Run the /opt/FastScale/etc/init.d/FSdhcpd stop command.

5. On the corporate DHCP server, update dhcpd.conf to add these options:

allow bootp;

allow booting;

next-server <IP address of the OS Provisioning Server>;

Configure TFTP

The OS Provisioning Server provides TFTP services that run on the provisioning network. You must configure the TFTP server to listen on the private OS provisioning network interface.

Procedure

1. On the OS Provisioning Server, log in as root.

2. Run ospctrl --showconfig.

The following results verify that the TFTP and Apache services are running.

TFTP - Configured on * - Running

Apache - Configured on * - Running

3. Run ospctrl --configure --privateip <IP Address>.

The configuration process runs. The IP address is 10.11.12.1.

Shutting down FStftpd: [ OK ]

Starting FStftpd: [ OK ]

TFTP - Configured on 10.11.12.1 - Running

Shutting down FSadmin: [ OK ]

Starting FSadmin: [ OK ]

Apache - Configured on 10.11.12.1 - Running

4. Run ospctrl --showconfig.

The following text appears when the TFTP and Apache services are running.

TFTP - Configured on 10.11.12.1 - Running

Apache - Configured on 10.11.12.1 - Running

What to do next

To install Windows distributions on target machines, you must create a Windows boot image and copy it to the OS Provisioning Server. See "Create a Windows Boot Image" on page 26.

Create a Windows Boot Image

Create a Windows boot image and copy it to the OS Provisioning Server. You create the image on a Windows 2008 or Windows 7 machine, and copy the files to the OS Provisioning Server.

26 VMware, Inc.

Installing, Configuring, and Upgrading the OS Provisioning Server and Components

Prerequisites

Verify that the Windows Automated Install Kit (WAIK) 2.0 is installed on the Windows machine on which you are creating the boot image.

Verify that the Windows machine on which you are creating the image, which is usually the VCM Collector, can access the OS Provisioning Server on the network.

On Windows 2008 machines, you run the command line options in this procedure as Administrator.

Procedure

1. On the OS Provisioning Server, copy /opt/FastScale/deployment to a directory on the Windows machines on which you are creating the boot image.

For example, c:\Program Files\osp.

2. From the Windows command line, change the directory to the location where you copied the deployment files.

For example, c:\Program Files\osp\deployment.

3. From the Windows command line, run bin\osp --osphome="c:<Path to OSP files> --

deploymenturl=<OS Provisioning Server Private IP Address> --waik=<Path to WAIK>".

Option Description

osphomee The path to the files copied from the OS Provisioning Server. For example,

c:\Program Files\osp\deployment. If you run the command from the

directory, you can use --osphome=.

deploymenturl

waik

The OS Provisioning Server's Private Interface IP Address. The default

configuration is 10.11.12.1.

Path to the Windows AIK files. For example, "c:\Program Files

(x86)\Windows AIK".

4. When the preinstallation environment and boot configuration are created, copy the directories from the WindowsAIK machine to the OS Provisioning Server.

From Windows AIK Machine To OS Provisioning Server

[path]\deployment\output\Boot /FSboot/

[path]\deployment\output\windows\amd64\winpe.wim /FSboot/windows/amd64/

[path]\deployment\output\windows\x86\winpe.wim /FSboot/windows/x86/

What to do next

Copy the VCM certificate to the OS Provisioning Server to ensure the successful installation of your Linux/ESX distributions. See "Copy the VCMCertificate to the OS Provisioning Server for Linux

Provisioning" on page 27.

Copy the VCMCertificate to the OS Provisioning Server for Linux Provisioning

VMware, Inc. 27

vCenter Configuration Manager Installation and Getting Started Guide

Prerequistes

Ensure that you have access to the VMware_VCM_Enterprise_Certificate_*.pem file in the \Program Files (x86)\VMware\VCM\CollectorData folder on the VCM Collector.

Procedure

1. Copy the VCM certificate, VMware_VCM_Enterprise_Certificate_*.pem, to the OS Provisioning Server/opt/FastScale/var/fsadmin/basic/ directory.

What to do next

Configure the secure Stunnel communications between the OS Provisioning Server and the VCM Collector. See "Configure OS Provisioning Server Integration with the VCM Collector" on page 28.

Configure OS Provisioning Server Integration with the VCM Collector

The integration between VCM and the OS Provisioning Server uses Stunnel to establish secure communication between and the SOAP services of the two components.

Prerequisites

Ensure that all private keys are RSA keys.

Ensure that certificates are created or obtained, and copied to the required locations using industry best practices.

On the Collector, copy the certificate to c:\Program Files (x86) \VMware\VCM\Tools\sTunnel\certs\vcm_stunnel_cert.pem.

On the Collector, copy the private key to c:\Program Files (x86)\VMware\VCM\Tools\sTunnel\key\vcm_stunnel_pk.pem.

On the OS Provisioning Server, copy the certificate to /opt/FastScale/var/certs/vcm_stunnel_ cert.pem.

Verify that all directories where these keys and certificates are stored are secured.

Procedure

1. "Configure Stunnel on the OS Provisioning Server" on page 29.

Stunnel is used to establish secure communication between VCM and the OS Provisioning Server SOAP services. On the OS Provisioning Server, copy the certificates to the locations specified in the stunnel.conf file and configure Stunnel to ensure that the connection on the OS Provisioning Server is operational.

2. "Configure Stunnel on the VCM Collector" on page 30.

The VCM Collector installation process installs Stunnel files that are used to establish secure communication between VCM and the OS Provisioning Server SOAP services. Configure Stunnel to ensure that the connection on the Collector is operational.

3. "Confirm Stunnel Configuration" on page 32.

Confirm that Stunnel communication between the OS Provisioning Server and the VCM Collector is configured and active before you provision target machines.

28 VMware, Inc.

Installing, Configuring, and Upgrading the OS Provisioning Server and Components

Configure Stunnel on the OS Provisioning Server

Prerequisit es

Review the VCM Stunnel certificate validation chain described in /opt/FastScale/etc/stunnel.conf.

Procedu re

1. On the OS Provisioning Server, log in as root.

2. Place the VCM Stunnel certificate validation chain in /opt/FastScale/var/certs.

All of the files in this directory are owned by root and have permissions of -rw-r--r--.

The Stunnel configuration file on the OS Provisioning Server is located in

/opt/FastScale/etc/stunnel.conf.

; stunnel configuration file for server proxy

; Some performance tunings

socket = l:TCP_NODELAY=1

socket = r:TCP_NODELAY=1

; debug = 7

cert = /opt/FastScale/var/certs/service.pem

key = /opt/FastScale/var/certs/private/service.key

; Either CAfile or CAPath, but not both, should be defined

; CAfile = /opt/FastScale/var/certs/ca-cert.pem

; Certificate Authority directory

; This is the directory in which stunnel will look for certificates when using the verify.

; Note that the certificates in this directory should be named

; XXXXXXXX.0 where XXXXXXXX is the hash value of the DER encoded subject of the

; cert (the first 4 bytes of the MD5 hash in least significant byte order).

; The hash can be obtained with the command: openssl x509 -noout -in cert.pem -hash

CApath = /opt/FastScale/var/certs

client = no

foreground = no

output = /opt/FastScale/logs/stunnel.log

pid = /opt/FastScale/logs/stunnel.pid

[fsmesgds]

VMware, Inc. 29

vCenter Configuration Manager Installation and Getting Started Guide

accept = 40610

connect = localhost:21310

; Authentication stuff

verify = 3

[fsrepods]

accept = 40607

connect = 127.0.0.1:21307

; Authentication stuff

verify = 3

3. Run the service FastScale restart command to restart Stunnel.

What to do next

After you configure the Stunnel on the OS Provisioning Server, you must configure the Stunnel communication on the VCM Collector. See "Configure Stunnel on the VCM Collector" on page 30.

Configure Stunnel on the VCM Collector

Prerequisit es

Secure the VCM Stunnel certificate and the VCM Stunnel private key according to your corporate best practices.

Verify that the [C:]\Program Files (x86)\VMware\VCM\Tools\sTunnel\certs\ directory exists on the Collector. If the directory does not exist, create it.

Verify that the [C:]\Program Files (x86)\VMware\VCM\Tools\sTunnel\key\ directory exists on the Collector. If the directory does not exist, create it.

Procedu re

1. On the Collector, place the VCM Stunnel certificate in [C:]\Program Files (x86)\VMware\VCM\Tools\sTunnel\certs\vcm_stunnel_cert.pem.

2. Place the VCM Stunnel RSAprivate key in [C:]\Program Files (x86)\VMware\VCM\Tools\sTunnel\key\vcm_stunnel_pk.pem.

3. Place the OS Provisioning Server Stunnel CA certificate validation chain in the files and directory specified in the stunnel.conf file.

The VCM Stunnel configuration file on the VCM application server is [C:]\Program Files

(x86)\VMware\VCM\Tools\stunnel.conf.

cert = C:\Program Files (x86)\VMware\VCM\Tools\sTunnel\certs\vcm_stunnel_

cert.pem

key = C:\Program Files (x86)\VMware\VCM\Tools\sTunnel\key\vcm_stunnel_pk.pem

;; Use stunnel in client mode

client = yes

30 VMware, Inc.

+ 242 hidden pages